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Introduction 

The introduction and acceptance of television has stimulated the need 
to discover by objective means how television can be used in military 
installations. The desirability of obtaining experimental evidence 
of the value of television for training was recognized by the U. S. 
Naval Training Device Center several years ago. Now a series of reports 
have been completed. 

This volume combines and summarizes these reports to show areas for 
successful application of television to training and areas requiring 
further research. The following blue pages give a concise summary of 
the results. The numbers in parentheses refer to the report from -which 
the information was derived. Reference to the specific details of the 
experiment will indicate the extent to which the findings are appli- 
cable. 

Summary of results 

1. Television has proved to be as effective or mor-e effective 
than regular instruction. 

2. The high initial costs, large personnel requirements, diffi- 
cult operation and maintenance requirements and complicated program- 
ming have restricted its use. 

Recommendation 



Since television is a good instructional media it is important to dis- 
cover whether it can be made acceptable and compete effectively with 
other instructional media. Tills can be done by considering its cost, 
effectiveness, personnel requirements, programming requirements and 
its acceptance by all persons involved* Television systems employing 
a minimum of equipment and personnel should mo?, t. nearly meet these 
requirements. Television should be used in those other instances 
where the unique characteristics of television make its use clearly 
advisable regardless of costs and operational problems. 



Lor an C. Twyford, Jr., Ph.D. 

Acting Head, Communication Psychology Division 



Clifford P. Seitz, Ph.D* 

Head, Human Engineering Department 



SUMMARY OF INSTRUCTIONAL TELEVISION RESEARCH REPORTS 



A. LIVE INSTRUCTIONAL TELEVISION FOR THE CLA SSROOM 



1. glfectiveness, A television program can be at least as effective as compa 
rable means of instruction (l;76-02-S2, U76-02-S3). 

2, Acceptance Television instruction is well liked (i*76-02-S2, 530-01-1). 
Well prepared programs were highly acceptable after an eight week' period 
of television training (i*76-02~S3), 



3. 



Mass i raining Television is a feasible and effective means for instruct- 
ing widely separated groups (li76-02-S2, i;76-02-S3). 



Principal Problems (U76-02-S2 ) 



a. Procurement and training of personnel for planning and producing 
television lessons. 

b. Procurement of television equipment and personnel to maintain it. 

Retention of Learning Iviost learned material was retained over a six week 
period (Itf6-02-S37. 

^ • Level of Ins truction All grades of personnel learned from television 
programs CU76—02--S3 J. 

7. Novelty Effect In 1950 trainees said that the television instruction they 
received was more effective than the average training film, ibis instruc- 
tion was carefully prepared, skillfully presented and the trainees tried 
to learn (V76-02-S3 ) . 

effec tive Presentations Items that were explicitly covered were well 
learned. Sketchily treated items were not learned (1+76-02-S3) . 

9* Dramatic or Factual Learning occurred when specific information was pre- 
sented. Little learning occurred from dramatic or situational presen- 
tations (U76-02-S3). 



10. Intro du c t i ons Speeches by high-ranking officials were with few exceptions 
too long, quite boring, poorly reproduced and detracted from the instruc- 
tional value of the programs (V76-02-S3) . 

11. Screen Size Twelve to twenty inch television screens were said to be 
adequate by trainees (U76— 02— S3 ) . 

12. Ap plicability A criteria check list has been developed to determine courses 
of instruction which are suited for television instruction (530-01-1). 

13 « Single Camera One television camera will fulfill most military training 
needs but for reliability and flexibility, two cameras are more desirable 
( 530 - 01 - 1 ). 




Instructors Qualified instructors can be trained to teach bv television 
in a relatively short time (530-01-1). 
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Minimum Equipment Television 
equipment gives the greatest 
( 530 - 01 - 1 ). 



experience has indicated that a 
training per dollar expended for 



minimum of 
televising 



l6* Courses effective television training has been carried out in larce number 
of subject areas (530-01-1). 

17. Films Films are effective on television (530-01-1, 1^76-02-32, U76-02-S3). 

lQ* Television Recordings (kinescopes) Film recordings of television programs 
are a valuable by-product (530-01-1, 20-TV-l, 1^76-02-32). 

15* Mobile Television A mobile television studio and, associated equipment can 
be used to present and record television programs. 



B. FIIM RECORDINGS OF TELEVISION PROGRAMS 



1. Effectiveness Film recordings oi television programs (kinescopes) are 
very satisfactory for military training' even though picture quality may be 
poor (20-TV-l, U76-02-S2). 

2. Use Kinescope recordings were recommended for training instructors, dupli- 
cating lessons, disseminating new developments and as a substitute for * 
instructional films (530-01-1). 

3. Color Color, unless it is essential to the subject being taught, does not 
increase the effectiveness of television training (20-TV-l). 

C. SPECIALIZED TELEVISION APPLICATIONS 

1* Training Device Viewing Training devices may be televised to a larger 
„ group thai can normally see them (530-01-1). Thirty-one principles for 
improving visibility have been discovered (20-TV-2). 

2* Security Applications Television signals containing classified information 
can be transmitted to the classroom over a closed circuit cable Q476-O2-S3 ) . 

3* Critical Factors Television e:xpense and labor can be more easily justified 
when the training situation is dangerous or mass training is essential 
( 530 - 01 - 1 ). 

D. INCIDENT IAL TELEVISION INSTRUCTION (Audience may or may not watch) 

I* Dramatiz at io ns Dramatic treatments brought about less learning than other 

.types of treatment (U76-02-S2). 

2. Attitudes Polls have shown a definite acceptance by civilians of programs 
dealing with book reviews, social problems, history, lectures, etc. 
( 530 - 01 - 1 ). 
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INTRODUCTION 

This technical supplement to Report NAVEXOS 850-2 presents in more detail 
the findings summarized in that report. It is concerned with an experimental deter- 
mination of the effectiveness of television training compared with other forms of 
training for Naval Air Reservists. The work herein reported was conducted under 
Contract N7onr-47602, and represents one phase of a pioneering study of training 
by television conducted at the Special Devices Center over the period October 1948 
to February 1951. 
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QUESTIONS TO BE ANSWERED 

This study was undertaken to obtain answers to the following questions: 

(1) Can television be used effectively for conveying information to widely 
separated groups? 

(2) Do groups receiving instruction by television learn as much as groups 
given traditional classroom instruction? 

(3) Is television instruction favorably received by reservists? 

(4) Are recordings of instructional telecasts effective when later projected 
as sound moving pictures ? 

(5) What problems would be encountered in using television for rapid, mass 
training? 



DESIGN OF THE EXPERIMENT 

The basic design of the experiment involved the following steps, as outlined in 
Figure 1: 

(1) Selecting topics for instruction: eight were designed for ref resher train- 
ing of officer pilots, and eight for basic training of enlisted airmen. 

(2) Developing a lesson plan for each of these topics. 

(3) Preparing tests for administration before and after each instructional 
session, using the lesson plans as a guide. 

(4) Preparing television instructional sessions based on the lesson plans. 



l 






(5) Broadcasting television presentations to groups of Naval Air Reservists 
in classrooms at three Naval Air Training Stations. 

(6) Preparing kinescope recordings of the television programs as they were 
being broadcast, and presenting copies of these recordings as sound movies 
to groups of reservists in classrooms at three additional Naval Air Train- 
ing Stations. 

(7) Submitting the lesson plans from which the television scripts had been 
developed to local instructors at three other Naval Air Training Stations, 
for their use in preparing instruction on the designated topics. Instructors 
presented lessons to reservists at their own stations. 

(8) Administering tests based on the lesson plans to all groups immediately 
before and immediately after instruction. 



DESIGN OF EXPERIMENT 
PROCEDURE FOR ONE UNIT OF INSTRUCTION 
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Figure 1 




NAR TECHNICAL TRAINING OFFICER ADVISORY BOARD 

A Naval Advisory Board was established to insure that the television training 
programs would comply with the training requirements of the Chief, Naval Air 
Reserve Training. This group consisted of: 

Lt. Cdr. Frank Adams; Alternate, Lt. Cdr, Joseph Rowland, Naval Air Station, 

New York, N.Y. 

Lt. Cdr. Patrick J. Finneran; Alternate, Lt. Cdr. Howard Payne, Naval Air 

Station, Anacostia, D.C. 

Lt. Cdr. Dean Rumbold, Naval Air Station, Willow Grove, Pa. 

Lt. William Sparks; Alternate, Lt. (jg) T.E. Glass, Naval Air Station, Squantum, 

Mass. 

This advisory group was designed to function as follows: (a) Plan topics to be 
treated in the series of Naval Air Reserve training programs, (b) Designate the 
material to be presented in each television program. This information was then to 
be submitted to lesson plan writers (Lewis Merkel, A/E-2 andLt. Cdr. Bert Shields), 
(c) Review the completed lesson plans and indicate any changes required to ensure 
adherence to CNAResTra requirements. After revision, the lesson plans would be 
turned over to script writers who were to prepare the television scenarios. 

In addition to the above functions, the Naval Advisory Board was to suggest for 
the various programs likely instructors from their home stations who would be 
screened for suitability; arrange classroom space for reservists viewing the pro- 
grams; and set aside suitable periods in the training schedule for pre- and post- 
testing the subject matter of each telecast. 

SUBJECTS 

The subjects were officer pilots and enlisted airmen of the Naval Air Reserve. 
As part of their regular training sessions, these men were assigned to participate 
in the experimental sessions. Trainees at Willow Grove, Pennsylvania; Floyd Bennett, 
New York; and Anacostia, Washington, D. C., received instruction by television. 
Trainees at Dallas, Texas; Olathe, Kansas; and Glenview, Illinois, received instruc- 
tion by means of the television recording (kinescope). Trainees at Akron, Ohio; 
Minneapolis, Minnesota; and Grosse He, Michigan, were taught by local instructors. 
At each station, approximately 40 men participated in each lesson; consequently, 
data are available on each lesson for 100 to 120 men taught by television, 100 to 120 
taught by kinescope, and a like number taught by local instructors. The exact number 
of trainees from whom complete records were obtained is reported in Table 1. 
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TABLE 1 



OFFICERS 

NUMBER OF TRAINEES FROM WHOM COMPLETE RECORDS WERE OBTAINED 
FOR EACIi LESSON, AND EACH TYPE OF INSTRUCTION 

NUMBER 



TITLE OF LESSON 


GROUP 


PARTICIPATING 




Television 


119 


Civil Air Regulations 


Kinescope 


116 




Local Instructor 


110 




Television 


93 


Aerology I 


Kinescope 


120 




Local Instructor 


120 




Television 


128 


Aerology II 


Kinescope 


120 




Local Instructor 


115 




Television 


113 


Navigation Review 


Kinescope 


111 




Local Instructor 


114 




Television 


129 


Radio Aids to Navigation 


Kinescope 


119 




Local Instructor 


120 




Television 


66 


Engineering 


Kinescope 


101 




Local Instructor 


120 




Television 


100 


High Altitude Flying 


Kinescope 


101 




Local Instructor 


120 




Television 


108 


Cross Country Flight 


Kinescope 


115 




Local Instructor 


116 



* K 
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TABLE 1 (Cont) 



ENLISTED AIRMEN 

NUMBER OF TRAINEES FROM WHOM COMPLETE RECORDS WERE OBTAINED 
FOR EACH LESSON, AND EACH TYPE OF INSTRUCTION 



TITLE OF LESSON 


GROUP 


NUMBER 

PARTICIPATING 




Television 


121 


Theory of Flight 


Kinescope 


120 




Local Instructor 


113 




Television 


122 


Line Safety 


Kinescope 


120 




Local Instructor 


115 




Television 


118 


Line Servicing 


Kinescope 


115 




Local Instructor 


118 




Television 


119 


Ordnance and Gunnery 


Kinescope 


111 




Local Instructor 


119 




Television 


124 


Weights and Balances 


Kinescope 


120 




Local Instructor 


120 




Television 


123 


Squadron Organization 


Kinescope 


112 




Local Instructor 


117 




Television 


70 


Survival and Safety 


Kinescope 


119 




Local Instructor 


120 


Jet Engines 


Television 


123 




Kinescope 


108 




Local Instructor 


114 




Attrition due to failure to complete tests or to sign booklets was extremely low 
amounting to only six-tenths of one per cent of the number of subjects participating 

in the experiment. 

At each station, diff er ent personnel participated in the successive lessons because 
of training schedules which called for training duty only once a month. Because of 
wide geographical scatter and the continually changing personnel, no procedures for 
selecting subjects in advance of the experimental sessions were feasible. 

The nine stations participating in the experiment were selected upon recommen- 
dation by the Naval Advisory Board. Each station had a well -developed reserve training 
program, and it was anticipated by the advisory group that trainees at all of these 
stations would be at approximately the same stage of training. Tests given immedi- 
ately before the experimental sessions showed that the groups subjected to the three 
instructional procedures earned mean scores which were, on the whole, quite com- 
parable. Such initial inequalities in knowledge as were found between instructional 
type groups were compensated for by appropriate statistical procedures (analysis of 

co-variance technique). 



Instructors for the television sessions were selected from personnel at Floyd 
Bennett, Willow Grove, andAnacostia Naval Air Training Stations or attached to the 
Special Devices Center. Officers gave instruction to the pilots, and Chief Petty 
Officers or technicians instructed the enlisted airmen. Within each group of in- 
structors there was considerable variation in the amount of teaching experience and 
in the degree of mastery of the subject matter taught. However, as a result of the 
search for outstanding instructors, the average instructor appearing on the television 
programs probably had been more successful as a teacher and knew more about his 
specialty then would the average instructor in the usual teaching situation at a Naval 
Air Training Station. 

Classroom instruction at Akron, Minneapolis, andGrosse lie Naval Air Training 
Stations was given by local instructors. The Commanding Officer and the Technical 
Training Officer at each of these stations was aware that comparisons were to be 
made between the outcomes of teaching by their personnel and television instruction. 
Consequently, it is highly probable that the local instructors were at least as care- 
fully selected (and perhaps more carefully) for the experimental teaching assignments 
as they would have been for normal teaching assignments. 



INSTRUCTORS 
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LESSONS TAUGHT 



The following is a complete list of the programs presented and the length of the 
instruction period for each. 



PROGRAM 



LENGTH OF 
INSTRUCTION PERIOD 



Refresher Training for Officers 



1. Civil Air Regulations: Procedures under instrument 

flight rules and visual flight rules 

2. Aerology I: Theory of weather changes affecting flight 

3. Aerology IT: Interpretation of weather maps and teletype 

weather sequence reports 

4. Navigation Review: Planning and plotting a flight 

5. Radio Aids to Navigation: Recent developments in radio 

ranges, direction finding, and landing systems 

6. Engineering: Pre- flight check, technical orders, and 

proper loading of aircraft 

7. High Altitude Flying: Training for safety and survival 

of the pilot at high altitudes 

8. Cross Country Flight: A simulated flight under marginal 

weather conditions 



45 minutes 
1 hour 

1 hour 
1 hour 

1 hour 

1 hour 

1 hour 

1 hour 



Basic Training for Enlisted Airmen 



1. Theory of Flight: Elementary aerodynamics 

2. Line Safety, Aircraft Handling, and Pre-Flight Check 

3. Line Servicing: Fueling and securing of aircraft 

4. Ordnance and Gunnery: Small arms and range procedure; 

flexible and fixed aerial gunnery 

5a. Weights and Balances: The optimal loading of aircraft 

5b. Squadron Organization: Types of squadrons and typical 
ratings in a sample squadron organization 

6. Survival and Safety: Use of life raft, emergency sig- 

nalling equipment, etc., following ditching aircraft 
at sea 

7. Jet Engines: Fundamental principles of the structure 

and operation of jet engines 



45 minutes 
1 hour 
1 hour 

1 hour 
40 minutes 

20 minutes 

1 hour 
1 hour 
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8. Nuclear Physics: A theoretical discussion of recent 
developments in atomic physics. No lesson plan 
was made available for use in preparing tests for 
this lesson; consequently, this session was not 
formally evaluated. It was presented only to the 
television group. 1 hour 



LESSQN PLANS 

Lesson plans for each subject except the one on ’’Nuclear Physics” were prepared 
by a group of naval officers assigned to cooperate in planning the instruction. These 
lesson plans included references to appropriate publications which would be helpful 
in preparing the instruction. A list of available films and other training aids was also 
included in each lesson plan. 

The "Nuclear Physics” session was planned and presented by Dr. Clark Goodman 
of the Massachusetts Institute of Technology, but no lesson plan was made available 
for use in preparing tests; consequently, this session was not formally evaluated. 




Figure 2 

ORGANIZATION CHART 
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PRODUCTION OF TELEVISION SESSIONS 

Programs originated in the television studio of the Special Devices Center and 
were produced and directed by personnel of the Television Section of the Special 
Devices Center. The Evaluation Group played no part in the production of the in- 
structional sessions, and operated independently of the Television Section. Figure 
2 presents an organization chart for the Television Section. 

VIEWING CONDITIONS IN TELEVISION CLASSROOMS 

Sixteen-inch television receivers were used, two to a classroom, and ten men 
were assigned to each receiver. The seating arrangement is shown in Figure 3. 
Microphones connected to telephone circuits made it possible for trainees to present 
questions at stated periods to the television instructor. 





VIEWING DISTANCE FROM TELEVISION SCREEN: 

1st ROW - 4 FEET 
2nd ROW - 8.5 FEET 
3rd ROW - 13 FEET 

SEATING WAS ARRANGED SO THAT NO ONE VIEWED 
THE TELEVISION SCREEN AT AN ANGLE GREATER 
THAN 35 DEGREES. 





TALK-BACK 

MICROPHONE 

O 



AUDIO 

SYSTEM 



Figure 3 




DIAGRAM OF SEATING ARRANGEMENT 
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EVALUATION PROCEDURES 



1. Tests. From each lesson plan, multiple choice questions were prepared. Samples 
of items are given below. 

Aerology II 

When a pilot sees a thunderstorm while in flight, there are several facts he should 
use in judging the direction in which it is moving. One of these is that 

1. the anvil and " shelves” protrude farthest ahead of the storm. 

2. In the United States, the storm generally moves in a direction opposite to 
that of the gradient wind. 

3. the roll cloud usually develops in the rear of the storm cloud. 

4. horizontal lightning streaks occur chiefly in the front portions of thunder- 
storms, with vertical streaks in the rear portions. 

Crosscountry Flight 

Normally, upon closing a VFR flight plan from a non-military base, a new flight 
plan may be filed at the same time only if the pilot 

1. intends to depart within the hour. 

2. plans to clear for a military base. 

3. intends to continue under VFR. 

4. notifies CAA. 

High Altitude Flying 

The purpose of breathing pure oxygen prior to flight is principally to 

1. acclimate the pilot to the peculiarities of oxygen breathing. 

2. prevent nitrogen bubble formation. 

3. test the adequacies of the equipment. 

4. increase rate of blood circulation thereby facilitating oxygen transport 
to the tissues. 

Survival an d Safety 

The Navy type multiplace raft should be boarded from 

1. the front. 

2. the rear. 

3. either front or rear. 

4. either side. 



Line Safety 

Pre -flighting a plane means 

1. a quick check of fuel and engine, 

2. a routine check before each flight. 

3. a detailed daily check on each plane. 

4. loading the plane for flight. 

Jet Engines 

The fuel nozzles in the combustion chamber of a jet engine are arranged in rings 

to 

1. cool the fuel. 

2. distribute the fuel evenly in the combustion chamber. 

3. save space. 

4. prevent forward burning into the compressor. 

From the pool of items prepared for a particular lesson, approximately thirty 
items were selected to make up a pretest to be given before the instruction. All of 
the items in the pretest were repeated in the final test, and they were supplemented 
by an equal number of additional items selected to sample similarly the lesson plan 
content. The time limit for the pretests was fifteen minutes, while thirty minutes 
were alldwed for the final tests. These timelimits permitted almost all reservists 
to attempt all items, hence the tests were measures of power rather than of speed. 
The pretest was given immediately before the instructional session, and the final 
test was given immediately after it. This denied the trainees any opportunity to 
discuss the test items or program content among themselves before taking the final 
test. 

Because of the short period of time available for preparing items, it was not 
possible to try out tests in advance and to refine them by item analysis. The appro- 
priate difficulty level of items had to be established by the opinions of experts. 

Before the instructional period, the mean percentage of correct responses on 
the various pretests earned by the approximately 350 officers in the Television, 
Kinescope, and Local Instructor groups combined, ranged from 41.9 to 58.8 and 
centered about 49.1. There were no zero scores, but one or two officers approached 
perfect scores on some of the pretests before instruction. It must be borne in mind 
that the officer programs were intended to constitute " refresher” training rather 
than instruction in new areas. After instruction, the mean percentages of correct 
responses on the final tests ranged from 57.8 to 65.1, and averaged 61.7. 
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Mean percentages of correct responses on the various pretests earned before 
instruction by the approximately 350 enlisted airmen in the Television, Kinescope, 
and Local Instructor groups averaged 46.1 and ranged from 36.6 to 55.3. There were 
no zero scores, but there were many low scores. After instruction, the mean per- 
centages of correct responses on the various final tests ranged from 52.8 to 78.0 
and centered about 69.0. A few individuals approached perfect scores on several of 

tll6 tfiGtS. 

Estimated reliability coefficients for each of the final tests were obtained by 
computing the correlation between scores on split-halves of each test for the Tele- 
vision group and applying the Spearman-Brown formula. For the officer final tests, 
the reliability coefficients ranged from .51 to .79. The reliability coefficients for 
the enlisted airmen final tests varied from .65 to .82. All reliability coefficients are 
reported in Table 14, in Appendix A. The estimated reliability coefficients of the 
tests are lower than would be considered desirable for individual prognosis, but the 
tests yield sufficiently stable measures for making comparisons between groups. 

Scoring of Tests. Incomplete test papers were discarded, and tests were scored 
only if a complete pretest could be matched with a complete final test for the same 
individual. Only six-tenths of one per cent of the papers marked were discarded for 
incompleteness. From scoring keys prepared with the aid of the Naval Advisory 
Board, tests were scored and independently rescored. The score on each test was 
the number of correctly marked items. 

Test Administration in the television classrooms was supervised by members 
of the Evaluation Project staff. Tests for other groups were supervised by local 
training officers. Prior to the first lesson at each station, a member of the Project 
“staff conferred with the training officer and emphasized the necessity.for safeguarding 
the tests and adhering closely to instructions. Excellent cooperation was secured and 
inspection visits showed seals on test envelopes unbroken up to the time they were to 

be used. 

2 Procedure for Compa ring Types of Instruction. To permit a comparison of the 
learning outcomes of television instruction with the learning effected by the usual type 
of reserve training, the prepared lesson plans were sent to three additional Naval Air 
Training Stations (Akron, Minneapolis, and Grosse lie) with a directive specifying that 
local instructors be assigned to teach the topics covered by the sixteen lesson plans to 
groups similar to the groups participating in the television phase. The time allowance 
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for each topic was the same as that used in telecasting. Local instructors were advised 
that the learning accomplished by their classes would be compared with the learning 
accomplished by students receiving television instruction. It is likely, therefore, that 
each local instructor exerted at least as much time and effort (and perhaps more) in 
preparing his presentation as he would expend on an average lesson. Students taught 
by the local instructors were also informed that they were participating in an experi- 
mental comparison between traditional instruction and television instruction. Pretests 
and final tests , identical with those used with the television classes , were administered 
before and after each lesson. 

Recordings were made of each program as itwa_s being telecast. Prints of these 
kinescope recordings were sent to another group of thre Naval Air Training Stations 
(Dallas, Olathe, Glenview) with instructions that they be projected as sound movies 
for comparable groups of reservists during their regular training periods. The train- 
ees were aware that they were viewing kinescope recordings of television programs, 
because the prints included opening and closing station announcements. They were aHo 
told that they were participating in an experimental comparison of kinescope in- 
struction and television instruction. The same pretests and final tests given to the 
other groups were administered to these trainees. 

3, Other Evaluative Evidence for the Television Programs. In addition to test data, 
three other types of information regarding the television instructional sessions were 
gathered. 

Comments by Students. At the conclusion of each television lesson, trainees were 
invited to write in their test booklets any comments, favorable or unfavorable, they 
desired to make regarding the program just viewed. 



Comments by .Staff of Evaluation Project and by Naval Advisory Board. One or 
more staff members of the Evaluation Project viewed each session of television 
instruction as it was received in the teleyision classrooms at Floyd Bennett, Willow 
Grove, and Anacostia Naval Air Stations. Viewing conditions were systematically 
observed and recorded. Observations were also made of the apparent reaction of the 
viewing groups to the programs. 

Following eompletion of the series of television lessons, the Naval Advisory Board 
met with the staff of the Evaluation Project, and the programs were systematically dis - 
cussed from the point of view of their adequacy as reserve training sessions. 



Analysis of Program Content, From the kinescope recordings, the types and 
amounts of instructional and non- instructional activities included in each television 
program were determined. 



TREATMENT OF RESULTS 

1. Test data. To compare the learning accomplished by the three types of instruc- 
tion, two measures were used. The first of these compared the improvement in score 
when the pretest items were repeated immediately after instruction. This measure 
was advantageous because it was based on identical tests given just before and im- 
mediately after instructional sessions. 

The second measure of the effectiveness of the three instructional procedures 
was based on a comparison of relative status on the double-length final tests, with 
statistical adjustment for such inequalities in knowledge as were found among groups 
before instruction. The advantage of this index is its more extensive, hence, more 
reliable, sampling of knowledge of the subject matter included in the various lessen 
plans. 

For the eight officer programs, there were consequently sixteen comparisons to 
be made between the outcomes of television instruction and teaching by local in- 
structor; sixteen comparisons to be made between the effectiveness of teaching by 
local instructor and by kinescope instruction; and, finally, sixteen comparisons be- 
tween the outcomes of television instruction and kinescope instruction. 

For the eight topics taught to the enlisted men, only fifteen comparisons were 
possible between each pair of instructional procedures, since the pretest and final 
test for the brief "Squadron Organization" lesson were identical. 

2. Comments by Trainees. From 36 per cent to 73 per cent of the officers and from 
44 per cent to 65 per cent of the enlisted airmen participating in each television lesson 
wrote some comment in the test booklets. Remarks of individuals varied from a few 
lines to rather lengthy statements. Remarks were transcribed verbatim from the 
booklets and aggregated 83 pages of single-spaced typewritten material. After a 
careful reading of the comments, tentative categories were adopted for tabulating the 
frequency of recurring remarks. Following several revisions of these categories, it 
was found possible to tabulate substantially all of the pertinent comments in 24 cate- 
gories. The broad headings under which these categories were developed dealt with 
the following: 



a. Subject matter and organization of material 

b. Adequacy of television reception and classroom conditions 

c. Instructors 

d. Tests 

3 . Comments by Staff of Evaluation Project and by Naval Advisory Board. Reports 
on the adequacy of video and audio reception made by staff members of the Evaluation 
Project in each classroom for each program were studied to determine whether 
reception of television instruction in any classroom had been seriously affected b$ 
transmission or reception difficulties. Such technical deficiencies as were reported 
were minor in character, and no television class had to be eliminated from consid- 
eration because of poor reception. 

Discussions of the television lessons by the Naval Advisory Board were sum- 
marized by staff members of the Evaluation Project. One type of summary related 
to the specific programs, and a second consisted of recommendations applicable to 
future television instruction. 

4 . Analysis of Program Content. A preliminary analysis of the television record- 
ings revealed that a minimum of 15 categories was required to describe adequately 
the types of activities included in the various programs. The percentage of program 
time devoted to each of these 15 categories was computed. The more effective and 
less effective television programs were compared to determine whether large or 
small amounts of specified activities characterized good or poor programs. 



RESULTS 

1 . TEST RESULTS . Tables 15 to 30, in Appendix B, presentthe differences between 
mean adjusted scores on pretests repeated after instruction and also differences be- 
tween final test mean adjusted scores for (a) the television group compared with the 
local instructor group, (b) the television group compared with the kinescope group, 
and (c) the kinescope group compared with the local instructor group. Each of the 
differences reported in these tables is accompanied by the standard error of the 
difference, thet-ratio of the difference compared to its standard error, and a verbal 
interpretation of the significance of the difference between means. 

Table 2 summarizes the interpretations of the significance of differences between 
specified officer groups on pretest items repeated after instruction, with adjustment 
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TABLE 2 



SUMMARY OF INTERPRETATIONS OF DIFFERENCES BETWEEN SPECIFIED OFFICER 
GROUPS ON PRETEST ITEMS REPEATED AFTER INSTRUCTION, WITH ADJUSTMENT 

FOR INITIAL INEQUALITIES 



Title of Session 


Television 
Compared to 
Local Instructor 
(LI) 


Telev is ion 
Compared to 
Kinescope 
(K) 


Kinescope 
Compared to 
Local Instructor 


1. Civil Air Regulations 


Equivalent 


Equivalent 


Equivalent 


2. Aerology I 


LI superior 


Equivalent 


LI superior 


3. Aerology 11 


TV superior 


Equivalent 


K superior 


4. Navigation Review 


TV superior 


Equivalent 


Equivalent 


5. Radio Aids to Navigation 


TV superior 


Equivalent 


K superior 


6. Engineering 


Equivalent 


Equivalent 


LI superior 


7. High Altitude Flying 


TV superior 


Equivalent 


K superior 


8. Cross Country Flight 


Equivalent 


Equivalent 


Equivalent 



NUMBER OF COMPARISONS INDICATING SPECIFIED CONCLUSIONS 



TV superior to Local Instructor 4 

TV equivalent to Local Instructor 3 • 
TV inferior to Local Instructor 1 



TV superior to Kinescope 0 

TV equivalent to Kinescope 8 

TV inferior to Kinescope f' 



Kinescope superior to Local Instructor 
Kinescope equivalent to Local Instructor 
Kinescope inferior to Local Instructor 



3 

3 

2 



(by means of the analysis of co-variance technique) for such initial inequalities as 
were found. Table 3 summarizes the interpretations of differences between final test 
mean scores of the various officer groups after adjustment for initial inequalities. 
Tables 4 and 5 present corresponding summaries of the interpretations of differences 
between specified enlisted airmen groups. 

In comparing the outcomes of two instructional procedures, if the difference in 
mean adjusted scores was so small that it might be expected to occur by chance more 
than five times in 100 repetitions of the experiment, the difference was indicated as 
insignificant and the two procedures were judged, for the given instance, as equivalent 
in effectiveness. A difference was considered significant only if the statistical 
analysis indicated that such a difference might be caused by chance factors two or 
fewer times in 100 repetitions of the experiment. Where such significant differences 
between the effects of instructional procedures were found (two per cent level of 
confidence, or more reliable), the procedure associated with the higher mean score 
was judged to be more effective, or superior to, the other procedure. The interpre- 
tation,, ” probably significant,” was attached to those differences which did not reach 
the two per cent level of confidence, but which were clearly above the five per cent 
confidence level. 

Compensation for such initial inequalities between groups as were found, was 
accomplished by utilizing the analysis of co-variance technique. In order to place 
all comparisons on the same basis, the co- variance technique was applied even in 
those instances where groups did not differ significantly in initial status. 

Figures 4 to 19, inAppendixC, present graphically the following scores for each 
group for each lesson: 

a. The mean pretest score. 

b. The obtained (that is, unadjusted) mean score on pretest items repeated 
after instruction. 

c. The mean obtained score on final test. 

A statistically sophisticated inspection of the mean obtained scores as repre- 
sented in these graphs would lead to substantially similar over-all conclusions re- 
garding the relative effectiveness of the three instructional procedures as were 
obtained from the more refined statistical analyses reported herein. 

Results from the pretest items repeated after instruction and from the complete 
final test were in harmony with each other. In no instance was an indication of superi- 
ority or inferiority for one of two instructional procedures yielded by data from 
pretests repeated after instruction contradicted by a reverse finding from the data 
for the complete final test. There were, of course, instances where a significant 
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TABLE 3 



SUMMARY OF {INTERPRETATIONS OF DIFFERENCES BETWEEN MEAN FINAL TEST 
SCORES OF SPECIFIED OFFICER GROUPS, WITH ADJUSTMENT 

FOR INITIAL INEQUALITIES 



Title of Session 


Television 
Compared to 
Local Instructor 
(LI) 


Television 
Compared to 
Kinescope 
(K) 


Kinescope 
Compared to 
Local Instructor 


1. Civil Air Regulations 


Equivalent 


TV probably 
superior 


Equivalent 


2. Aerology I 


Equivalent 


Equivalent 


Equivalent 


3. Aerology II 


TV superior 


Equivalent 


1C superior 


4. Navigation Review 


Equivalent 


Equivalent 


EciiiK'-ilent 

A 


5. Radio Aids to Navigation 


TV superior 


• 

Equivalent 


K s»?pc;ior 


6. Engineering 


LI superior 


Equivalent 


LI superior 


7. High Altitude Flying 


TV superior 


Equivalent 


K superior 


3. Cross Country Flight 


TV superior 


Equivalent 


Equivalent 



NUMBER OF COMPARISONS INDICATING SPECIFIED CONCLUSIONS 



TV superior to Local Instructor 4 

TV equivalent to Local Instructor 3 
TV inferior \0 Local Instructor 1 

TV superior to Kinescope 1 

TV equivalent to Kinescope 7 

TV inferior to Kinescope 0 



Kinescope superior to Local Instructor 
Kinescope equivalent to Loc ^Instructor 
Kinescope inferior to Local Instructor 



3 

4 
1 
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TABLE 4 



SUMMARY OF INTERPRETATIONS OF DIFFERENCES BETWEEN SPECIFIED ENLISTED 
AIRMEN GROUPS ON PRETEST ITEMS REPEATED AFTER INSTRUCTION, WITH 

ADJUSTMENT FOR INITIAL INEQUALITIES 



Title of Session 


Television 
Compared to 
Local Instructor 
(LI) 


Television 
Compared to 
Kinescope 
(K) 


Kinescope 
Compared to 
Local Instructor 


1. Theory of Flight 


Equivalent 


Equivalent 


Equivalent 


2. Line Safety 


LI superior 


TV probably 
superior 


LI superior 


,3. Line Servicing 


LI superior 


Equivalent 


LI superior 


4. Ordnance and Gunnery 


TV superior 


Equivalent 


K superior 


5a Weights and Balances 


TV superior 


Equivalent 


K superior 


5b Squadron Organization 


TV superior 


TV superior 


Equivalent 


6. Survival and Safety 


TV superior 


Equivalent 


K superior 


7. Jet Engines 


TV superior 


Equivalent 


K superior 


8. Nuclear Physics 


* 


* 


* 



NUMBER OF COMPARISONS INDICATING SPECIFIED CONCLUSIONS 



TV superior to Local Instructor 5 

TV equivalent to Local instructor 1 
TV inferior to Local Instructor 2 



TV superior to Kinescope 2 

TV equivalent to Kinescope 6 

TV inferior to Kinescope 0 

Kinescope superior to Local Instructor 
Kinescope equivalent to Local Instructor 
Kinescope inferior to Local Instructor 



* No tests were administered. 
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TABLE 5 



SUMMARY OF INTERPRETATIONS OF DIFFERENCES BETWEEN MEAN FINAL TEST 
SCORES OF SPECIFIED ENLISTED AIRMEN GROUPS, WITH ADJUSTMENTS 

FOR INITIAL INEQUALITIES 



Title of Session 


Television 
Compared to 
Local Instructor 
(LI) 


Television 
Compared to 
Kinescope 

(K) 


Kinescope 
Compared to 
Local Instructor 


1, Theory of Flight 


Equivalent 


Equivalent 


Equivalent 


2. Line Safety 


LI superior 


TV superior 


LI superior 


3. Line Servicing 


LI superior 


Equivalent 


LI superior 


4. Ordnance and Gunnery 


TV superior 


Equivalent 


K superior 


5a Weights and Balances 


Equivalent 


Equivalent 


Equivalent 


5b Squadron Organization 


* 


* 


* 


6. Survival and Safety 


TV superior 


Equivalent 


K superior 


7. Jet Engines 


TV superior 


TV probably 
superior 


K superior 


8. Nuclear Physics 


** 


** 


** 



NUMBER OF COMPARISONS INDICATING SPECIFIED CONCLUSIONS 

TV superior to Local Instructor 3 

TV equivalent to Local Instructor 2 
TV inferior to Local Instructor 2 

TV superior to Kinescope 2 

TV equivalent to Kinescope 5 

TV inferior to Kinescope 0 

Kinescope superior to Local Instructor 3 

Kinescope equivalent to Local Instructor 2 

Kinescope inferior to Local Instructor 2 

* Identical with data in Table 4, since all items in pretest were repeated in final test. 

** No tests administered. 
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superiority of one instructional procedure indicated by final test data for a given 
comparison was paralleled by an ins»gnificant superiority of tho same procedure 
according to data from pretest items repeated after instruction. 

All of the comparisons between television instruction and teaching by local 
instructors reported in Tables 2, % 4, and 5 have been further consolidated for ready 
interpretation in Table 6, which reports the percentage of comparisons indicating 
that television was (a) superior to, (b) equivalent to, or (c) inferior to teaching by the 
local instructors. This table shows that television instruction was found to be more 
effective than teaching by local instructors in half of the comparisons made, and 
was equally effective in a substantial proportion of the comparisons. 



TABLE 6 



CONDENSED SUMMARY OF INTERPRETATIONS OF DIFFERENCES 
IN LEARNING EFFECTED BY TELEVISION AND BY LOCAL INSTRUCTORS 



Television superior to Local Instructor 
Television equivalent to Local Inc.iuctor 
Television inferior to Local Instructor 



Percentage of Comparisons 
Indicating Specified Conclusions 


Programs for 


Programs for 


Officers 


Enlisted Airmen 


50 


53 


38 


20 


13 


27 



Table 7 summarizes all the interpretations of differences in learning effected 
by kinescope presentation compared with teaching by local instructors which were 
reported in Tables 2, 3, 4, and 5. When the kinescope recordings were presented 
as sound movies , they wei found to be more effective than teaching by local instructors 
for 38 per cent of the officer comparisons and for 47 per cent of the enlisted airmen 
comparisons. Greater learning was brought about by the local instructor in only 19 
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per cent of the comparisons for pilot programs and 27 per cent for the enlisted 
programs. Kinescope presentations were as good as or better than teaching by local 
instructor in three-quarters of the comparisons made. 

TABLE 7 

CONDENSED SUMMARY OF INTERPRETATIONS 
OF DIFFERENCES IN LEARNING EFFECTED BY KINESCOPE 
AND BY LOCAL INSTRUCTOR PRESENTATIONS 



Percentage of Comparisons 
Indicating Specified Conclusions 





Programs for 


Programs for 




Officers 


Enlisted Airmen 


Kinescope superior to Local Instructor 


38 


47 


Kinescope equivalent to Local Instructor 


44 


27 


Kinescope inferior to Local Instructor 


19 


27 



Table 8 presents a condensed summary of interpretations of comparisons 
between learning resulting from viewing television lessons directly with that re- 
sulting from viewing recordings of the lessons. (This table is based on material 
extracted from Tables 2, 3, 4, and 5.) In this table only, for the sake of simplicity, 
three comparisons earlier placed in the interpretative category "probably significant" 
because they were significant at approximately the three per cent confidence level 
were here merged with comparisons in the "significant" category. For officer 
programs, the two procedures were shown to be equally effective by 94 per cent of 
the comparisons made. In the case of the programs for the enlisted airmen, tele- 
vision and kinescope recordings were indicated to be equally effective in 73 per cent 
of the comparisons, and television was indicated as more effective in 27 per cent 
of the evaluations. 
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TABLE 8 



CONDENSED SUMMARY OF INTERPRETATIONS OF DIFFERENCES 



IN LEARNING EFFECTED BY TELEVISION AND KINESCOPE PRESENTATIONS 




Percentage of Comparisons 




Indicating Specified Conclusions 




Programs for 


Programs for 




Officers 


Enlisted Airmen 


Television superior to Kinescope 


6 


27 


Television equivalent to Kinescope 


94 


73 


Television inferior to Kinescope 


0 


0 



2 ACCEPTABILITY OF TELEVISION INSTRUCTION . The comments which trainees 
were invited to write in the test booklets after each program indicated that television 
instruction was , in general, very favorably received by them. Some details of various 
programs were given SDecific criticisms, but there were only a few complaints that 
any lesson as a whole was poor or dull. There was considerable similarity between 
the comments of officers and enlisted airmen regarding the two series of television 
programs. 



Comments of Officers 



Summary of comments by officers, pr esented in Table 9. From 36 to 73 per cent 
of the officers participating in each television lesson wrote some comment in the 
test booklets. Remarks varied from a few lines to rather lengthy statements. When 
transcribed from the test booklets, the verbatim reports for the eight sessions 
aggregated 43 pages of single- spaced typewriting. Except for the large number of 
comments stating that the programs were good in general, there were more adverse 
comments regarding details than favorable comments. Apparently, when a feature 
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of the program was of acceptable quality, no comment was made by most of the 
viewers. 

After a careful reading of the comments, tentative categories were adopted for 
tabulating the frequency of recurring remarks. Following several revisions of these 
categories, it was found possible to tabulate substantially all of the pertinent com- 
ments by use of the response categories included in Table 9. 

Some individuals commented on only a single feature of a program, whereas 
others wrote remarks which touched upon a number of aspects of the television 
lesson. Consequently, in Table 9 it will be found that the sum of the frequencies of 
the various comments for a particular program will considerably exceed the number 
of individuals commenting upon that session. 

Some complained that early programs were paced too fast. The first of four broad 
categories under which remarks were classified was "Subject Matter and Organiza- 
tion of Material" and sub-heading A dealt with "Quantity and Rate of Presentation." 
As reported i;i Table 9, Section I-A, from 15 to 20 of the individuals who viewed 
any one of the first four programs expressed the belief that too much material had 
been included in these programs, or thatthe material had been presented too rapidly. 
Of those who participated in the fifth television lesson, Radio Aids to Navigation, 
40 complained that too much content was included, or that the pace was too fast. 
These 40 men represented 31 per cent of those viewing the program in formal 
classes, and constituted 43 per cent of those who wrote any comment on the lesson. 
The pace of the last three programs was apparently more acceptable, for only one 
to three individuals reported that too much material was included in any of these 
instructional sessions. Only one or two officers complained that any program was 
presented too slowly. 

Comments on the quality of the programs were generally favorable, b ut the number 
of favorable comments varied from program to program. Section I-B of Table 9 
reports frequency of comments regarding the quality of each program. There is a 
considerable variation from one program to another in the number who reported 
that "the program was interesting and/ or of value," frequencies ranging from 12 to 
56. There were also appreciable numbers who commented that given programs had 
merit or showed promise, but who cited specific shortcomings. These remarks are 
tabulated in Table 9, Section I-B, under the heading "Programs good in general, but 
limited on one or more counts." The specific limitations cited included such items 

as the following: 
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Too little student- instructor contact 
Poor organization and/or choice of material 
Lack of textbooks or other printed material 
Type and amount of humor 
Important items should be repeated 



The first program in the series was the one most enthusiastically received. Ninety- 
nine per cent of those commenting on it said it was effective or interesting, or good 
in general with some stated limitations. This generally favorable reception may 
have resulted in part from the publicity preceding the first program and the 
enthusiasm expressed by the speakers who opened this session. For the remaining 
programs, from 50 to 74 per cent of the comments were favorable either with or 
without limitation. The sixth program, devoted to "Engineering," elicited the 
smallest per cent of favorable comments. It is possible that scheduling this session 
for the Labor Day weekend may have made it less acceptable to the reduced number 
of officers who participated in it. 

Few asserted traditional classroom instruction or training films would have been 
preferred. The third line of Section I-B of Table 9 reports that from three to six 
individuals commented that traditional classroom instruction or training films 
would have been as good or better than the television instruction in the second, 
third, fourth and fifth sessions. This represented from 3 to 8 per cent of those 
making any comment on these programs, or from 2 to 6 per cent of those who viewed 

them. 

Only a few officers considered any complete program unsatisfactory. There were 
no general adverse comments made regarding the first and seventh programs 
devoted respectively to Civil Air Regulations and High. Altitude Flying. For the 
other six programs, from one to eight officers expressed the opinion that the pro- 
gram as a whole was poor. The Navigation Review and the Cross Country Flight 
programs received respectively 5.5 and 3.5 times as many favorable comments, 
limited or unlimited, as they did unfavorable comments. The remaining four pro- 
grams which received any adverse comment as a whole were given from 9 to 20 
times more favorable comments than unfavorable ones. Majority opinion is clearly 
favorable to the television presentations, but the ratio of favorable to unfavorable 
comment varies considerably from one program to another. 
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Only one or two officers challenged the accuracy of any material in a television 
session. Section I-C of Table 9 shows that one or two officers challenged the 
accuracy of some of the material presented in five of the eight programs. The 
points raised were all highly specific and related to brief sequences in the programs. 
In some instances, the charged ’’errors" represented momentary lapses on the part 
of the instructor and in other cases were due to failure to qualify statements suf- 
ficiently. It is interesting to note that the accuracy of the eight hours of instruction 
received only seven challenges from the 856 participants in the television classes. 

Some programs were considered too theoretic al by about one-sixth of viewing group. 
From one-fifth to one- seventh of those commenting on Aerology I, Radio Aids to 
Navigation and Engineering sessions expressed the belief that more emphasis should 
have been given to practical aspects of those subjects and less to theory. As shown 
in Section I-D of Table 9, similar remarks were made by smaller numbers of 
individuals regarding three additional programs. 

One- hour program apparent ly was not considered too long in most instances, but 
th ere are exceptions. The two Aerology sessions were judged to be too long by 12 
and 18, respectively^ of the officers participating in these instructional sessions. 
These frequencies represented 17 per cent and 22 per cent of the numbers making 
any comment on these two programs, or roughly one out of seven of those who 
viewed the programs. Five of the six remaining sessions were also judged to be too 
long by from one to three of the participants. A television instructional session one 
hour in length is apparently acceptable to a great majority of those who took part in 
this study, but the character, pace or content of a teaching period may make it seem 
unreasonably long to an important minority. 

Only a few officers comme nted on visual aids. Section I-F of Table 9 presents a 
tabulation of comments on the visual aids utilized in the eight programs. From two 
to eight of the officers making comments on the various programs considered that 
there was insufficient visualization or that the visual aids were poorly chosen and/or 
poorly presented. An equal or smaller number of participants in each program 
considered the visual aids to be good, the number ranging from zero to nine. 



Opportunity for the student t o communicate with the instructor was found inadequate 
by 10 per cent of the officers viewing three programs . The opportunity for student- 
instructor communication was judged to be inadequate by front 9 to 14 of the 
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participants in the Aerology I, Aerology II, and Radio Aids programs. For each of 
five of the remaining programs, two or three individuals considered communication 
between student and instructor to be inadequate. Considering the eight programs as 
a whole, 44 participants considered there was insufficient opportunity to ask questions 
while only 6 considered the question periods adequate. 

Very few complain ed about the television picture or viewing conditions^ Section II 
of Table 9 presents tabulations of comments on Picture Quality, Audio Quality, and 
Classroom Conditions. The number of comments in each category for a single 
program seldom exceeds two, and is frequently zero. There were few complaints 
for any program regarding either video or audio quality. A few more officers 
considered the seating arrangement poor than considered it good. There were 
only five complaints for the eight programs about poor ventilation. Classroom 
noise during tests and inadequate lighting each were the object of a few complaints. 
From the small number of comments and the division of opinion among them, it is 
concluded that video quality, audio quality, and classroom conditions were acceptable 
to most of the participants in these programs. 

Each instructor was considered inadequate in one or more respects by from one to 
eighteen viewers. There were four times as many adverse comments about the 
instructors as there were favorable comments. Some of the 53 adverse comments 
for the eight programs were unqualified, while others referred to specific charac- 
teristics such as delivery, voice, attitude, and apparent knowledge of subject matter. 
Section III of Table 9 presents the data regarding comments on instructors. 



Tests claimed to include topics not treated in the television session. The fourth 
and last of the categories under which comments were tabulated refers to "Tests.” 
From zero to three individuals considered the tests used for a particular program 
to be "difficult, and/or ambiguous," but such comments averaged only one per pro- 
gram. There was considerably more complaint that the test items for a particular 
session covered some material which was not treated in the television instruction. 
These comments averaged between eight and nine per program. Most of these 
were legitimate complaints, for in each lesson plan some material was either not 
treated in the television session or was given only casual treatment. Since the 
tests were designed to cover the content indicated by the lesson plan rather than 
only the material treated in the television session, any appreciable departure from 
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the lesson plan by the television instructor resulted in justifiable complaints that 
some test items were not touched upon in the given instructional session. 



Comments of Enlisted Airmen 



About half of the viewers wrote some comment regarding the television sessions. 
The enlisted airmen accepted the invitation to write comments, favorable or un- 
favorable, regarding the television programs only a little less frequently than did 
the officers. From 44 to 65 per cent of the participants in the television classes 
for enlisted airmen wrote comments upon the various programs. When transcribed 
from the test booklets, these comments occupies* 40 pages of single-spaced type- 
written material. These comments were edited and classified under headings 
substantially similar to those developed for classifying the comments of pilots, and 
Table 10 reports the frequency of each type of comment for each of the television 
lessons. The two parts of the fifth session dealing with ’’Weights and Balances" and 
"Squadron Organization" are here treated as a unit since there was no consistent 
differentiation of comments for the two topics. 

Some programs were considered paced too fast by a small number of viewers. Ten 
participants in the Ordnance and Gunnery Session considered that too much material 
had been presented or that the rate of presentation was too rapid. These constituted 
13 per cent of those who commented on this program or 8 per cent of those who 
viewed it in the formal classes. Seven viewers, or 10 per cent of those making 
comments, had a similar complaint regarding the Jet Engines session. For three 
of the remaining sessions, from one to four individuals commented that too much 
material had been introduced, and for two of the sessions, Line Safety and Line 
Servicing, no complaints about pace were recorded. 



A majority of the trainees reacted favorably to each of the television sessions, but 
the frequency of favorable comment varied considerably from program to prog ram. 
In Section I-B of Table 10 are reported the number of comments indicating that the 
programs vvere effective and/or interesting, or that they were good in general, but 
limited in one or more respects. From 50 to 82 of the comment on the separate 
sessions were favorable without qualification. If the qualified and unqualified re- 
marks are considered together, the range of generally favorable comments for the 
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various programs is from 57 to 93 per cent of the number of persons making any 
comment on the given sessions. 

The Survival and Safety program received the smallest percentage of generally 
favorable comments, namely 57 per cent. This sessionwas presented on the Labor 
Day weekend, and it is possible that the smaller number of men participating in this 
session were less favorably inclined toward the training situation because of the 
holiday, it will be recalled that the Engineering program for officers was also 
scheduled on the holiday weekend, and it received the smallest proportion of favorable 
comments of any of the officer series of programs. 

Four programs elicited over 80 per cent of generally favorable comments. 
Line Safety (83 per cent), Line Servicing (93 per cent), Ordnance and Gunnery (84 
per cent) and Jet Engines (82 per cent). 



Traditional classroom procedures or training films were preferred by few, except 
for one program. Slightly more than ten per cent of those commenting on the "Weights 
and Balances” and "Squadron Organization” presentations asserted that traditional 
classroom treatment and/or training film would be as good or better than the tele- 
vision instruction. From one to three trainees made similar assertions regarding 
each of the three other programs. 

There were only three comments on the series of seven sessions expressing 
the view that practice rather than theory should be emphasized (Section I-C, Table 
10). There was apparently no strong feeling among the trainees that the instructional 

sessions were too theoretical. 

One pro gram was considered too long by seven trainees. A total of only 18 comments 
on the seven sessions (Section I-D) expressed the belief that the television lessons 
were too long, and seven of these referred to one session, namely "Weights and 
Balances” and "Squadron Organization.” 

Insufficient visualization was claimed for the first two programs. Section I-E of 
Table 10 reports tabulations of comments on visual aids. On the first two programs, 
18 per cent and 14 per cent, respectively, of those making comments asserted that 
there was insufficient visualization. For the remaining programs, from one to five 
individuals reported an inadequate amount of visual aids, these frequencies 
representing from three to eight per cent of those who wrote comments on the pro- 
grams concerned. Commendation of the visual aids used were given by from two 
to nine of the individuals commenting on five of the sessions. 
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Some trainees complained of the lack of opportunity to ask questions. Twenty- two 
of the remarks for the sev@n programs considered together, stated that there was 
insufficient opportunity to ask questions. Section I-F of Table 10 shows that from 
two to six such comments were made for each of six programs and these low fre- 
quencies represented 3 to 10 per cent of those commenting on the particular sessions. 

Viewing conditions were gen erally considered adequate. Section II of Table 10 shows 
that relatively few comments were made for any one program concerning Picture 
Quality, Audio Quality, or Classroom Conditions. In all, there were 19 statements 
that Picture Quality was poor opposed to 7 claims that it was good. Only 6 trainees 
commented on Audio Quality and these wtfre evenly divided between "good" and "poor." 
Under Classroom Conditions, only four remarks were tallied, and these all referred 

to poor ventilation. 

Few comments were made regarding the instructors. The number of trainees com- 
menting on the instructor for any program was relatively small (Section HI, Table 
10) and there was considerable variation from program to program in the ratio of 
"good" to "poor" ratings. For the first two programs, there were six times as 
many "poor" ratings as there were "good." Instructors for the third, fourth, and 
sixth sessions received more favorable than unfavorable comments, but for the fifth 
and seventh programs, this relationship was reversed. 

Some test items were claimed not to be treated in the television programs. There 
were only four complaints in all, stating that the tests were difficult and/or ambiguous 
and three of these related to the tests for a single session. A considerably greater 
number of complaints asserted that some items in the test were not covered in the 
television presentation. There were from one to eight such comments for each of 
the seven programs. This point was discussed in connection with the Officer pro- 
grams, and it was acknowledged that the criticism is a legitimate one in most 
instances. As the tests were based on the lesson plans, any marked deviation from 
the outlined content by the television producer or instructor will result in the failure 
to treat (or to treat adequately) one or more topics used as the basis for items 
included in the test. 

3. COMMENTS BY THE STAFF OF THE EVALUATION PROJECT AND BY THE 
NAVAL ADVISOPV BOARD . The comments of staff members of the Evaluation Group 
who viewed the lessons in the television classrooms at the various Naval Air Train- 
ing Stations indicated: 
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a. That on the whole, the television sessions were favorably received 
by the reservists and held their attention and interest. 

b. That while a number of graphics and models were employed, greater 
and more effective use of the visual aspects of television would have increased the 
effectiveness of the programs as training sessions. It appears probable that the 
television presentation of a speaker is more interesting to most observers than the 
radio presentation of the speaker’s voice. However, there can be little question that 
when an instructor is talking about an operating mechanism, it is more effective 
for learning to have the operating mechanism (or a model or a graphic of it) appear 
on the screen in place of the instructor’ s picture. In the absence of color television, 
ordinary training aids painted or printed in various colors cannot be used effectively. 
Sharply contrasting colors may have identical gray scale values and will be repro- 
duced similarly on the black and white television screen. Cut-away models of weapons 
or engines presented serious camera and lighting problems, and in most instances, 
were not reproduced with a fully satisfactory degree of clarity on the screens in the 
television classrooms. 

c. Tint in many instances, the television production staff apparently 
concentrated more on ’’interesting camera angles” and constantly changing screen 
presentations for the purpose of ’’relieving monotony,” than they did on the conveying 
of meanings. As a consequence, angle shots of a blackboard or a chairt frequently 
resulted in a distorted and difficult- to- comprehend picture. In other instances, 
charts or models were permitted to remain on the scree* tor too short a period to 
permit trainees to read or comprehend the meanings which were to have been 
conveyed. In still other instances, insufficient attention was devoted to density of 
ideas or to vocabulary burden. It was felt that application of appropriate psycho- 
educational techniques in planning and producing the instructional sessions would 
have resulted in more effective presentations . Effective television instruction makes 
demands upon the production staff which differ in some respects from the demands 
of commercial broadcasting. 

d. That the talk-back system designed to permit trainees to ask 
questions of the television instructor was relatively unsatisfactory. Technical 
operating difficulties limited the usefulness of the system, but these could be over- 
come under other circumstances. The basic difficulty was that the questions asked 
were, for the most part, trivial and could have been answered by the officer in 
charge of the television classroom. Some ’’screening” of questions is essential if 
optimal -use of network time is to be realized in resolving the difficulties experienced 
by the students in the television classrooms. 
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The principal recommendations of the Naval Advisory Board regarding future 
television training sessions were: 

a. That more and better visual training aids be employed. 

b. That a larger number of potential television instructors be made 
available to a screening board, so that the board could select for participation in 
the television sessions, instructors who will be free from peculiarities of speech 
or annoying mannerisms. 

c. That one hour of instruction with little or no student participation is 
excessively long. It was felt that a question or discussion period near the middle 
of the session with a second such period near the end of the hour would be desirable. 

d. That closed- circuit distribution of programs be considered. Broad- 
casting on channels open to the public introduced public relations problems which 
would be eliminated by use of closed circuits. Also, the use of public television 
channels appeared to place a greater strain on the instructors than if they knew their 
instructional performances would be viewed and heard only by reservists reached 
by closed circuits. 

e. That a training specialist be used to serve as over-all coordinator 
of all phases of the training sessions. In this way, training values would not be 
subordinated to production procedures. 



4. RELATION O F PROGRAM CONTENT TO PROGRAM EFFECTIVENESS . When 
the television recordings were analyzed to determine the relative amount of program 
time devoted to various activities, it was found necessary to recognize fifteen cate- 
gories under a total of 11 headings. The terms applied to these categories and brief 
* descriptions of the type of activity assigned to each category follow: 

Opening and closing music, station 
identification, introductory or clo- 
sing film which was irrelevant to 
the program proper, and intro- 
ductory remarks describing the na- 
ture of the program. 

Speeches which were unrelated to 
the subject matter of the program. 



1. OPEN AND CLOSE: 



2. SPEECHES: 
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3. ANIMATED INSTRUCTIONAL 
FILM WITH NARRATION ON 
SOUND TRACK: 

4. OTHER INSTRUCTIONAL 
FILM WITH NARRATION 
ON SOUND TRACK: 

5. ANIMATED INSTRUCTIONAL 
FILM WITH TV STUDIO 
NARRATOR: 

6. OTHER INSTRUCTIONAL 
FILM WITH TV STUDIO 
NARRATOR: 

7. INSTRUCTIONAL FILM 
WITHOUT NARRATION: 



8. ANIMATED ATMOSPHERIC 
FILM WITH TV STUDIO 
NARRATION: 



9. OTHER ATMOSPHERIC FILM 
WITH TV STUDIO NARRATION: 

10. ATMOSPHERIC FILM 
WITHOUT NARRATION: 



Animated film having meaning by 
itself, or amplifying the meaning 
conveyed by the sound track. 

All non- animated sound film con- 
veying meaning related to instruc- 
tion. 

Identical with Category 3, except 
that narration is by instructor or 
narrator in the television studio. 

Identical with Category 5, except 
that it includes only non- animated 
film. 

Silent film considered to convey 
meaning related to instructional 
aspects of program. 

Animated film conveying meaning 
only incidentally related to instruc- 
tion and accompanied by television 
studio narration. 

Identical with Category 8, except 
it includes only non- animated film. 

Film conveying meaning only inci- 
dentally related to instruction, and 
unaccompanied by any type of nar- 
ration. 



11. DIRECT NARRATION: All sequences of instruction not 

dramatized or accompanied by film. 



12. INSTRUCTIONAL DRAMATIZED 
ACTION WITH OFFSTAGE 
NARRATION: 

13. INSTRUCTIONAL DRAMATIZED 
ACTION WITH DIALOGUE: 



14. "FILLER” DRAMATIZED 
ACTION WITH DIALOGUE: 



15. QUESTION PERIOD: 



Role playing accompanied by off- 
stage narration considered to be 
meaningful as instruction. 

Role playing involving two or more 
people, one or more with speaking 
parts considered to have value as 
instruction. 

Same as Category 13, but non- 
instructional; apparently introduced 
for comic relief or change of mood. 

Periods during which the students 
were given an opportunity to inter- 
rogate the instructor, or, as on one 
program, a period during which the 
instructor questioned students iden- 
tified by seat number in a specified 
classroom. 



Table 11 presents the percentage of program time devoted to each of the 
specified activities for each of the television lessons. 

Table 12 (derivedfrom Tables 2, 3, 4, and 5) reports the number of comparisons 
of test results leading to the conclusion (a) that television was more effective than 
teaching by the local instructor, (b) that the television instruction was equivalent 
in effectiveness to that by the local instructor, or, (c) that the television instruction 
was less effective than that given by the local instructor. Comparison of the data 
of these two tables permits identification of the features associated with the tele- 
vision sessions which were more effective than teaching by local instructors and 
those features associated with television sessions that were less effective than 
teaching by local instructors. 

The amount of direct narration for those television programs unequivocally 
indicated as superior to classroom instruction averaged approximately one-third 
higher than for all other programs combined. For off icer programs (3) Aerology n, 
(5) Radio Aids to Navigation, (7) High Altitude Flying, both test comparisons 
indicated the greater effectiveness of television instruction and the average amount 
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TABLE 11 

IT IN TERMS OF PERCENTAGE OF PROGRAM TIME DEVOTED TO SPECIFIED ACTIVITIES 
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TABLE 12 



NUMBER OF COMPARISONS OF TEST RESULTS FOR EAC.l 
PROGRAM INDICATING SPECIFIED CONCLUSIONS REGARDING 
EFFECTIVENESS OF TELEVISION INSTRUCTION RELATIVE TO 
TEACHING BY LOCAL INSTRUCTORS 

INDICATED CONCLUSIONi 





OFFICER PROGRAM NUMBER 
12345678 


TV Superior to Local Instructor 


2 12 2 1 


TV Equivalent to Local Instructor 


2 1 1 1 1 


TV Inferior to Local Instructor 


1 1 




ENLISTED AIRMEN PROGRAM NUMBER 
1 2 3 4 5a 5b 6 7 


TV Superior to Local Instructor 


2 112 2 


TV Equivalent to Local Instructor 


2 1 


TV Inferior to Local Instructor 


2 2 



of program time devoted to direct narration was 76 per cent as shown in Table 13. 
The corresponding percentage for all other officer programs averaged 57 per cent. 
In the case of the enlisted programs, both test comparisons indicated greater 
effectiveness of television programs (4) Ordnance and Gunnery, (6) Survival and 
Safety, and (7) Jet Engines, and the sole comparison for Lesson 5b indicated the- 
same conclusion. The average percentage of program time given to direct narration 



o 

ERIC 



40 



for these more effective programs was 78 per cent, contrasted to 57 per cent for 
all the other enlisted programs. Table 11 shows that the sixth officer program, 
Engineering, used an unusually large amount of non-animated instructional film 
with narration on the sound track (38.9 per cent of program time )« Table 12 shows 
that the two test comparisons indicated this program to be either equivalent to the 
local instruction or inferior to the local instruction. In contrast, the fifth officer 
program, Radio Aids to Navigation, devoted 22.8 per cent of program time to 
animated instructional film with narration on sound track, and both test comparisons 
indicated that television instruction was more effective than teaching by the local 
instructor. 

Two of the television programs for enlisted airmen, (2) Line Safety and (3) 
Line Servicing, were markedly less effective than local instruction. The analysis 
of program content showed that these two programs were the only two which used 
dramatized action with off-stage narration. Forty- three per cent of the Line 
Servicing program, and 21 per cent of the Line Safety program were devoted to this 
type of presentation. Table 5 also shows that the question periods for these two 
programs were twice as long as for any other sessions, accounting for 14 per cent 
of the hour for each lesson. 

The noted relationships between program effectiveness and proportion of 
program time devoted to specified activities are suggestive rather than conclusive 
in character. However, they do indicate the need for planned and systematic variation 
in the type of treatment accorded given topics. 



TABLE 13 



AVERAGE PERCENTAGE OF PROGRAM 
TIME DEVOTED TO DIRECT NARRATION 



Officer 

Programs 



Enlisted 

Programs 



On television programs shown by all 
test evidence to be more effective 
than teaching by local instructor 



76 



78 



On all ether television programs 



57 



57 



PROBLEMS IN EXPLOITING TELEVISION FOR RAPID MASS TRAINING 



The principal problems which would be encountered in exploiting television 
instruction for rapid mass training are: 

1. Procurement or training of instructors. Currently, very few instructors 
have had any experience in teaching before television cameras. 

2. The training of personnel in special techniques to adapt production proce- 
dures to the requirements of mass training. Effective educational television sessions 
make demands upon production personnel which differ in important respects from 
the demands of commercial television entertainment sessions. 

3. Procurement of personnel who know howto develop television lesson outlines 
or scripts in a manner which would incorporate sound educational practices. 

4. Procurement of physical equipment for the studio and for television class- 
rooms. During an emergency period, procurement of electronic equipment offers 
serious problems unless plans are made in advance. 

5. Training or procurement of personnel to maintain equipment in proper 
working order. Electronic engineers and technicians require training and experience 
with television equipment if they are to operate effectively. 

6. Accession or construction of distribution facilities. For distribution of 
programs to widely separated stations, use of co-axial cable circuits or microwave 
relay networks would be required. Less serious distribution problems would be 
encountered if the programs were to be distributed to groups within a single station. 



SUMMARY 

This study can be summarized by brief answers to the questions stated at the 
beginning of this report, and repeated here for convenience. 

1. Q. Can television be used effectively for conveying information to widely 
separated groups? 

A. The data of this study show that television is a feasible and effective 
means of conveying instruction to classes at widely separated stations. 
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2. Q. Do groups receiving instruction by television learn as much as groups 
given traditional classroom instruction? 

A. For both officers and enlisted airmen, television instruction was found 
to be more effective than teaching by local instructors in half of the comparisons 
made, and it was equally effective to teaching by local instructors in an additional 
one-fourth of the comparisons. 

3. Q. Is television instruction favorably received by reservists? Do they like 
it as well as their usual training sessions? 

A. Comments written by trainees immediately following television sessions 
indicated that the television instruction was in general very favorably received. 
Some details of various programs were given specific criticisms, but there were 
only a few scattered complaints that any lesson as a whole was poor or dull. Very 
few of the participants in the television instruction sessions indicated that they 
believed either training films or ordinary classroom instruction were as good or 
better than television teaching. 

4. Q. Are recordings of instructional telecasts effective when later projected 
as sound moving pictures? 

A. Recordings of instructional telecasts were extremely effective when 
projected as sound moving pictures. For the officer programs, the kinescope 
recording was equivalent in effectiveness to the television session in 94 per cent 
of the comparisons made. In the case of the programs for the enlisted airmen, the 
kinescope recordings were indicated to be equivalent in effectiveness to the tele- 
vision sessions in 73 per cent of the comparisons made. Kinescopes were as good 
or better than teaching by local instructors in three-quarters of the comparisons 
made. Generalization of this finding should, however, be limited to recordings 
which retain the element of "freshness" or "timeliness." Because they were re- 
cordings of recent live programs, tlse kinescopes used in this study were, perhaps, 
given more favorable reception than might be accorded them at a much later date. 

5. Q. What problems would be encountered in using television for rapid, mass 
training? 

A. The principal problems are (a) those of procurement and training of 
personnel so that sound psycho-educational techniques may be utilized in planning 
and producing television lessons, (b) the procurement of physical equipment and 
the personnel to maintain it, and (c) acquisition or construction of distribution 
facilities. 
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RECOMMENDATIONS 



It has been demonstrated that effective teaching can beaccomplishedby instruc- 
tional sessions conveyed bytelevision towidely separated groups. It is recommended 
that further inquiries be initiated to determine: 



1. What kinds of subject matter can be best taught by television. 

2. In what training programs of the Armed Forces can television be used most 
economically and advantageously. 

3. How good instructors can be located, and how they can be developed as tele- 
vision instructors. 

4. What production techniques will place the least burden on the television 
instructor. 

5. What kinds of instructional treatment are best for various types of material. 

6. What amount of communication between the television instructor and trainees 
is essential for optimal results, and what types of communication are most satis- 
factory and economical. 
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APPENDIX A 



RELIABILITY COEFFICIENTS 




APPENDIX A 



TABLE 14 

ESTIMATED RELIABILITY COEFFICIENTS OF FINAL TESTS BASED ON 
SPLIT-HALF COEFFICIENTS CORRECTED BY THE SPEARMAN-BROWN FORMULA. 



OFFICER SESSIONS 

No. of Items No. of 

in Final Test Cases r ll 



Civil Air Regulations 


50 


119 


.51 


Aerology I 


60 


93 


.73 


Aerology II 


60 


128 


.71 


Navigation Review 


55 


113 


.79 


Radio Aids to Navigation 


54 


129 


.61 


Engineering 


60 


66 


.59 


High Altitude Flying 


50 


100 


.53 


Cross Country Flight 


49 


108 


.58 




ENLISTED AIRMEN 


SESSIONS 






No. of Items 


No. of 




p’ 


in Final Test 


Cases 


r ll 


Theory of Flight 


60 


121 


.76 


Line Safety 


70 


122 


.77 


Line Servicing 


65 


118 


.73 


Ordnance and Gunnery 


60 


119 


.68 


Weights and Balances 


39 


124 


.78 


Squadron Organization 


20 


123 


.68 


Survival and Safety 


60 


70 


.65 


Jet Engines 


50 


123 


.82 



V ? 
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APPENDIX B 



COMPARISON OF LEARNING BY GROUPS RECEIVING 
EACH TYPE OF INSTRUCTION 



Tables 15 to 30 



x' "\ 

s - „/ 

TABLE 15 

CIVIL AIR REGULATIONS 

COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated after Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S - E -DUf 


t 


Interpretation 


TV - Local Instr. 


0.1 


.28 


0.4 


Not Significant 


TV - Kine 


0.1 


.27 


0.4 


Not Significant 


Kine - Local Instr. 


0.0 


.28 


0.0 


Not Significant 



Differences between Adjusted Final Test Mean Scores 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E ’Diff 


t 


Interpretation 


TV - Local Instr. 


0.8 


.49 


1.6 


Not Significant 


TV - Kine 


1.0 


.48 


2.1 


Probably Significant 


Kine - Local Instr. 


-0.2 


| .49 


0.4 


Not Significant 



TABLE 16 
AEROLOGY I 

COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated after Instruction 



Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


1 

S,E 'Diff 


t 


Interpretation 


TV - Local Instr. 


-0.9 


.35 


2.6 


Significant 


TV - Kine 


0.2 


.33 


0.6 


Not Significant 


Kine - Local Instr. 


-1.1 


.33 


3.3 


Significant 



Differences between Adjusted Final Test Mean Scores 



Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E *Diff 


t 


Interpretation 


TV - Local Instr. 


0.7 


.65 


1.1 


Not Significant 


TV - Kine 


0.4 


.65 


0.6 


Not Significant 


Kine - Local Instr. 


0.3 


.61 


0.5 


Not Significant 
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TABLE 17 
AEROLOGY II 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 


Differences between Adjusted Mean Scores on Pretests Rei 


leated aftei 


r instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


E.E-Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


1.0 

-0.1 

1.1 


.36 

.35 

.36 


2.8 

0.3 

3.1 


Significant 
Not Significant 
Significant 


Differences between Adjusted Final Test Me. 


m Scores 




> Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E ’Diff 


t 


Interpretation | 


W - Local Instr. 
j TV - Kine 

1 Kine - Local Instr. 


1.7 

0.0 

1.7 


.66 

.65 

.87 


2.3 
0.0 
2.5 . 


Significant 
Not Significant 
Significant 


1 





TABLE 18 
NAVIGATION REVIEW 



comparison of learning by groups receiving each type of instruction 



Differences between Adjusted Mean Scores on Pretests Repeated after 

1 


Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E. D iff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


0.9 

0.6 

0.3 


.37 

.37 

.37 j 


2.4 

1.6 I 

0.8 j 


Significant 
Not Significant 
Not Significant 


Differences between Adjusted Final Test Me 


an. Scores 




Groups Compared 
lat minus 2nd 


Differences in 
Adjusted Means 


S * E *Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


0.6 

1.1 

-0.5 


.58 

.58 

.58 


1.0 

1.9 

0.9 


Not Significant 
Not Significant 
Not Significant 




TABLE 19 

RADIO AIDS TO NAVIGATION 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated afte 


r Instruction 


Groups Compared 


Differences in 


S.E...I.. 


t 


Interpretation 


1st minus 2nd 


Adjusted Means 


Diff 


• 




TV - Local Instr. 


1.5 


.33 


4.5 


Significant 


TV - Kine 


0.1 


.33 


0.3 


Not Significant 


j Kine - Local Instr. 


1.4 


.33 


4.2 


Significant 


Differences between Adjusted Final Test Mean Scores 




Groups Compared 


Differences in 


S.E. n . r . 


t 


Interpretation 


1st minus 2nd 


Adjusted Means 


Diff 






TV - Local Instr. 


2.4 


.52 


4.6 


Significant 


TV - Kine 


0.3 


.52 


0.6 


Not Significant 


Kine - Local Instr. 


2.1 


.54 


3.9 


Significant 



TABLE 20 
ENGINEERING 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated after Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E ‘Diff 


t 


Interpretation 


TV - Local Instr. 


-0.6 


.39 


1.5 


Not Significant 


TV - Kine 


0.2 


.40 


0.5 


Not Significant 


Kine - Local Instr. 


-0.8 


.34 


2.4 


Significant 



Differences between Adjusted Final Test Mean Scores 



Groups Compared 
1st minur 2nd 


Differences in 
Adjusted Means 


S,E, Diff 


t 


Interpreter" ' on 


TV - Local Instr. 


-1.9 


.69 


2.8 


Significant 


TV - Kine 


0.9 


.71 


1.3 


Mot Significant 


Kine - Local Instr. 


-2.8 


.61 


4.6 


Significant 



s. ^ 
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TABLE 21 

HIGH ALTITUDE FLYING 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated afte 


r Ins' _tion 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E-Diff 


t 


Interpretation 


TV - Local Instr. 


1.2 


.38 


3.2 


Significant 


TV - Kine 


0.2 


.40 


0.5 


Not Significant 


Kine - Local Instr. 


1.0 


.38 


2.6 


Significant 



Differences between Adjusted Final Test Mean Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E. D iff 


t 


Interpretation 


TV - Local Instr. 


2.3 


.54 


4.3 


Significant 


TV - Kine 


0.5 


.57 


0.9 


Not Significant 


Kine - Local Instr. 


1.8 


.54 


3.3 


Significant. 



TABLE 22 

CROSS COUNTRY FLIGHT 

COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated after Instruction 


Groups Compared 


Differences in 


S.E. n .„ 


t 


Interpretation 


1st minus 2nd 


Adjusted Means 


Diff 






TV - Local Instr. 


0,6 


.45 


1.3 


Not Significant 


TV - Kin<± 


0.2 


.45 


0.4 


Not Significant 


Kine - Local Instr. 


0.4 


.44 


0.9 


Not Significant 




Differences between Adjusted Final Test Mean Scores 


Groups Compared 


Differences in 




t 


Interpretation 


1st minus 2nd 


Adjusted Means 


Diff 






TV - Local Instr. 


1.3 


.45 


2.9 


Significant 


TV - Kine 


0.8 


.45 


1.8 


Not Significant 


Kine - Local Instr. 


0.5 


.44 


1.1 


Not Significant 
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TABLE 23 
THEORY OF FLIGHT 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



1 Differences between Adjusted Mean Scores on Pretests Re] 


peated afte: 


r Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


SE Diff 


t 


Interpretation 


TV - Local Instr. 


0.3 


.37 


0.8 


Not Significant 


TV - Kine 


-0.3 


.37 


0.8 


Not Significant 


Kine - Local Instr. 


0.6 


.37 


1.6 


Not Significant 


j Differences between Adjusted Final Test Mean Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E ’Diff 


t 


Interpretation 


TV - Local Instr. 


0.6 


.62 


1.0 


Not Significant 


TV - Kine 


-0.2 


.62 


0.3 


Not Significant 


Kine - Local Instr. 


0.8 


.63 


1.3 


Not Significant 



TABLE 24 
LINE SAFETY 

COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Ref 


teated aftei 


r Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E. D iff 


t 


Interpretation 


T7 - Local Instr. 
j TV - Kine 

1 Kine - Local Instr. 


-3.1 

0.8 

-3.7 


.39 

.38 

.39 


7.9 

2.1 

9.5 


Significant 
Probably Significant 
Significant 



Differences between Adjusted Final Test Mean Scores 



Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S ' E, Diff 


t 


Interpretation 


TV - Local Instr. 


-2.9 


.69 


4.2 


Significant 


TV - Kine 


2.3 


.69 


3.3 


Significant 


Kine - Local Instr. 


-5.2 


.69 


7.5 


Significant 
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TABLE 25 
LINE SERVICING 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Re; 


jeated after Instruction 


Groups Compared 
1st minus 2nd 


— 

Differences in 
Adjusted Means 


S.E-Diff 


t I 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


-3.5 

-0.5 

-3.0 j 


.35 

.35 

.35 1 


10.0 
1.4 
j 8.6 


Significant 
Not Significant 
Significant 


Differences between Adjusted Final Test Me 


an Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E-Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


-4.6 

-0.1 

-4.5 


.58 

.58 

.58 


7.9 

0.2 

7.8 


Significant 
Not Significant 
Significant 



TABLE 26 

ORDNANCE AND GUNNERY 

COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences 

Groups Compared 
1st ir’nus 2nd 



between Adiusted Mean Scores on Pretests Repeated alter Instruction 



TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 



Groups Compared 
1st minus 2rd 



TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 



Differences in 
Adjusted Means 



1.0 

'.3 

1.3 



S.E-Diff 


t 


Interpretation 


.38 


2.6 


Significant 


.38 


0.8 


Not Significant 


.38 


3.4 


Significant 



Differences between Adjusted Final Test Mean Scores 



Differences in 


S.E.—i.** 


t 


Interpretation 


Adjusted Means 


' 'Diff 






6.1 

O.i 

6.0 


.66 


9.2 


Significant 


.67 


0.1 


Not Significant 


.67 


9.0 


Significant 



TABLE 27 

WEIGHTS AND BALANCES 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated afte 


r Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E 'Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


1.2 

0.2 

1.0 


.34 

.34 

.34 


3.5 

0.6 

2.9 


Significant 
Not Significant 
Significant 


Differences between Adjusted Final Test Mean Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


SE Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


0.8 

1.0 

-0.2 


.58 

.58 

.58 


1.4 

1.7 

0.3 


Not Significant 
Not Significant 
Not Significant 



TABLE 28 

SQUADRON ORGANIZATION 



, COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Repeated aftei 


r Instruction 1 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


SE Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


0.9 

1.6 

-0.7 


.35 

.35 

.36 


2.6 

4.6 

1.9 


Significant 
Significant 
Not Significant 









TABLE 29 

SURVIVAL AND SAFETY 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Rej 


>eated aftei 


r Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E, Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


2.8 

0.3 

2.5 


.43 

.43 

.37 


6.5 

0.7 

6.8 


Significant 
Not Significant 
Significant 


Differences between Adjusted Final Test Me: 


in Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S.E-Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


6.5 
1.0 

5.5 


.70 

.76 

.65 


8.6 

1.3 

8.5 


Significant 
Not Significant 
Significant 



TABLE 30 
JET ENGINES 



COMPARISON OF LEARNING BY GROUPS RECEIVING EACH TYPE OF INSTRUCTION 



Differences between Adjusted Mean Scores on Pretests Rep 


eated after 


Instruction 


Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S ’ E ’Diff 


t 


Interpretation 


TV - Local Instr. 
j TV - Kine 

I Kine - Local Instr. 


5.7 
1.0 

4.7 


.51 

.52 

.53 


11.2 

1.9 

8.9 


Significant 
Not Significant 
Significant 


Differences between Adjusted Final Test Me 


an Scores 




Groups Compared 
1st minus 2nd 


Differences in 
Adjusted Means 


S,E ’Diff 


t 


Interpretation 


TV - Local Instr. 

TV - Kine 

Kine - Local Instr. 


9.3 

1.7 

7.6 


.80 

.82 

.83 


11.6 

2.1 

9.2 


Significant 
Probably Significant 
Significant 
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APPENDIX C 



MEAN OBTAINED SCORES OF THREE GROUPS ,„ T _ 
ON TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTIONS 



Figures 4 to 19 
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FIGURE 4 

MEAN OBTAINED SCORES OF THREE GROUPS ON "CIVIL AIR REGULATIONS" 
TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical Uce 

line in each case extends froia -is.u.w+id, u. 




Television 

presentation 

N.=tl9 



Classroom 
Teaching by 
Local Instructor 
N=110 



Kinescope 

presentation 

N=U6 



FIGURE 5 

MEAN OBTAINED SCORES OF THREE GROUPS ON "AEROLOGY I" TESTS 
ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates location <rf a mean on scale score 3 Hortaontal 










SCORES 
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figure c 

MEAN OBTAINED SCORES OF THREE GROUPS ON -AEROLOGY II" TESTS 

mean before and after instruction 

Each vertical line ^*'1 s' dTcI distribution ol scores, 

line in each case extends from -1 S.D. w> + l 




Television 

Presentation 

N=123 



Classroom 
Teaching by 
Local Instructor 
N=115 



Kinescope 

Presentation 

N=120 



SCORES 



FIGURE 7 

MEAN OBTAINED SCORES OF THREE GROUPS ON -NAVIGATION REVIEW- 
TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Farh vertical line indicates location ol a mean on scale of scores. Horizontal 
£e £ fach c£e ertends from -1 S.D. to + 1 3.D. of distribution of scores. 




SCORES 



82 



MEAN 



FIGURE 8 



OBTAINED SCORES O 
TESTS ADMINIST 



F THREE GROUPS ON "RADIO AIDS TO NAVIGATION 
EF ED BEFORE AND AFTER INSTRUCTION 



II 



Parh vertical line indicates location of a mean on scale of scores. Horizontal 
SS h fc ?2f <£e emends from -1 S.D. to + 1 S.D. of distribution of scores. 




SCORES 






FIGURE 9 

MEAN OBTAINED SCORES OF THREE GROUPS ON "ENGINEERING" TESTS 
ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates location of a mean on scale of score. 3. Horizontal 
line in each case extends from -1 S.D. to +1 S.D. of distribution of scores. 
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FIGURE 10 



MEAN OBTAINED SCOR ES OF THREE GROUPS ON "HIGH ALTITUDE FLYING- 
TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Etch vertical line indicates location of a mean on scale of scores. Horizontal 
line in*sach case extends from -1 S.D. to + 1 S.D. of distribution of scores. 
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FIGURE 11 

mean FL,0HT " 



F-nh vertical line indicates location of a mean on scale of scores. Horizontal 
ftae in eSse emends from -1 S.D. to +1 S.D. of distribution of seres. 
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FIGURE 12 



MEAN OBTAINED SCORES OF 
TESTS ADMINISTERED 



THREE GROUPS ON "THEORY OF FLIGHT 
BEFORE AND AFTER INSTRUCTION 



II 






10 



1 5 

i — 



20 



25 



30 
— r~ 



35 



40 



45 
T W7 



50 55 



Television 

Presentation 

N=121 



Kinescope 

presentation 

N=120 



-i u 

0 5 



30 Item Test Before (B) and After (A) Instruction 

fey 

b | — ste-‘ 

* — h- 

y*: 

I w$*- 



Classroom B 

Teaching by 
Local Instructor 
N=113 



B 



VW'.MTj- '.J;» 

^60 Item Final Test‘d 

: • x , <•' • • *£> .. ••• ;• • A. • . • • , •»' *"• ; 



60 






L W: ., 

[ill:' 



» « > *»■* 



fcT'f'*, :• ' 






10 



15 



20 



25 



30 



35 



40 



45 



50 



• v % 
' ■ A 
' ; 1 

> V J 



1 






55 



60 



SCORES 



FIGURE 13 

MEAN OBTAINED SCORES OF THREE GROUPS ON "LINE SAFETY*' TESTS 
ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates location of a mean on scale of scores. Horizontal 
line in each case extends from -1 S.D. to +t S.D. of distribution of scores. 
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35 Item Test Before (B) and After (A) Instruction 
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FIGURE 14 

MEAN OBTAINED SCORES OF THREE GROUPS ON "LINE SERVICING" TEST 
MEAN OBTA ; N d ^ n1stered before AND AFTER INSTRUCTION 

Each vertical line in'Ucates locatwn o^a mean on ^cale of ^ution ol scores, 
line in each case extends from -X S.D. to +1 s.u. u*a 
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FIGURE 15 

nu THREE GROUPS ON "ORDNANCE AND GUNNERY" 
MEAN ° B £ A g!Jg® dMINISTERED BEFORE AND AFTER INSTRUCTION 
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figure 1C 

MEAN OBTAINED SCORES OF THREE GROUPS ON "WEIGHT S AND BALANCES" 
TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates Ration of a mean on scale 

line in eici case extends from -1 S.D. to +i 01 U1JUIUUI 
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figure r? 

MEAN OBTAINED SCORES OF THREE GROUPS ON "SQUADRON ORGANIZATION- 
TESTS ADMINISTERED BEFORE AND AFTER INSTRUCTION 
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FIGURE li/ 

MEAN OBTAINED SCOR ES O F THR E E GROUPS ON "SURVIVAL AND SA FETY" TESTS 
ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates location of a mean on scale of scores. Horizontal 
line in each case extends from -1 S.D. to +1 S.D. of distribution of scores. 
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FIGURE 19 

MEAN OBTAINED SCORES OF THREE GROUPS ON "JET ENGINES" TESTS 
ADMINISTERED BEFORE AND AFTER INSTRUCTION 

Each vertical line indicates location of a mean on scale of scores Horizontal 
line in each case extends from -1 S.D. to +1 S.D. of distribution of scores. 
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INTRODUCTION 



This report is one of a series concerned with the evaluation of television as a 
medium for accomplishing rapid mass training. It is a technical supplement to the 
non-technical report on the same material, NAVEXOS P850-3 (SDC Report 476-02-2) 
"Training By Television: A Study In Learning And Retention." Both the non-tec - 
nical report and the present technical supplement deal with a study of learning and 
retention by Army Field Force Reservists to whom a series of eight one- our 
instructional sessions was conveyed by television. The instructional telecasts were 

transmitted over anetwork of ten commercial broadcasting stations, and they reached 

approximately 3000 reservists in 160 viewing groups. The work herein reported was 
conducted by the Television Evaluation Project, Department of Psychology, Fordham 
University, under Special Devices Center Contract No. N7onr-47602. 
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PURPOSE 



The purposes of this study were: 

l To S oek additional evidence regarding the feasibility of conducting reserve 
training by television and to determine the effectiveness of such traimng. 

2. To determine the extent to which material learned through television 
instruction is retained. 

3. To determine the acceptability of television instruction to enlisted and 
officer personnel. 

4 . To study the influence of the type of instructional treatment upon the amount 
of learning effected. 



PROCEDURE 



A. DESIGN OF THE EXPERIMENT 

Figh t one-hour lessons each concerned with the operation of an Army Division 
in a combat situation were telecast at weekly intervals to more than 3000 Army 
Field Force Reservists. These programs originated at the U. S. Navy Special 
Devices Center Television Studio, and were microwaved to CBS- TV, New York, an 
then telecast over ten stations in the eastern and north central portions of the United 
States as shown in Figure 1. Reservists assembled in 160 groups to view the pro- 
grams, and took tests immediately before and immediately after each instructional 

session. 

Faculty members of the Army Command and General Staff College at Fort 
Leavenworth prepared the basic plans for the lessons and cooperated with technical 
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NEW YORK 
WCBS 76 GROUPS 

Special devices center 

T.V STUDIO 

PHILADELPHIA 
WCAU 21 GROUPS 



BALTIMORE 
WMAR 12 GROUPS 

WASHINGTON 
WOIC 2 GROUPS 



Figure 1. C. B. S. Television Network Used to Transmit the Command Post Series 



writers at the Army Signal Corps Photographic Center in the development of scripts. 
A different team of writers and producers was responsible for the preparation and 
presentation of each of the eight programs . The Evaluation Group had no part in 
determining the content of any ssion or the treatment accorded any topic. 

Test questions were prepared by staff members of the Television Evaluation 
Project from tentative drafts of the scripts. The pretest and the final test for a 
given program included a group of identical questions, and learning resulting from 
that instructional session was measured by (a) increase in number of items cor- 
rectly answered by each individual and also (b) by the increase in percentage of 
correct responses to single items . In the pretest booklets for Programs 3 to 8, some 
questions were included which had been presented before and after earlier programs. 
Success in responding to these repeated items was compared to success in responding 
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to them immediately after the program in which treatment of the item had been 
included. In this way, measures of retention over periods of one to six weeks were 
secured. 

Kinescope recordings of the programs were carefully analyzed to determine 

(a) the explicitness of treatment of topics around which items had been written, and 

(b) the type or types of instructional treatment accorded each of the topics covered 
by a test item. The relationship between the explicitness of treatment of topics and 
gain in percentage of correct responses to test items related to those topics was 
studied. The influence of the type of instructional treatment given a topic upon the 
amount of learning was also investigated. 

Questionnaires were administered to trainees after the first and last programs 
to determine how acceptable they found television instruction. Experts in military 
and civilian training viewed each of the programs and wrote comments on prepared 
forms. 



B. THE PROGRAMS 



The series of eight one-hour television instructional sessions titled Command 
Post dealt with the problems encountered, and the activities engaged in, by a Division 
Commander, his staff, and his troops in repelling an invasion by an "aggressor” 
nation. The programs combined narration with numerous dramatic sequences 
exemplifying the activities of the Division Commander and his staff. Troop move- 
ments, convoys, armored vehicles, aircraft attacks, etc., were pictured at appropriate 
points in the programs by the use of motion picture film. The topics treated in the 
eight programs may be summarized briefly as follows: 



PROGRAM 1: ESTIMATE OF THE SITUATION 

A portrayal of the development of a coordinated estimate of 
the significance of aggressor activity by the Division Com- 
mander and members of his staff. 

PROGRAM 2: ADVANCE TO CONTACT 

An account of the activities involved when the Division makes 
its first contact with the enemy, including aircraft support, 
maintenance of security, protection against aggressor air 
force, etc. 



PROGRAM 3: THE DIVISION COMMANDER’S DECISION 

Presentation of the factors that' determine the decision of the 
Division Commander about place, type, and time of attack. 

PROGRAM 4: PLANNING THE ATTACK 

An account of the planning of maneuvers in the attack. This 

includes the location of the reserve, and the direction of the 

attack, while the Division is maintaining contact with the 

enemy forces. 

PROGRAM 5: THE LOGISTICAL PLAN 

A treatment of the supply and service activities that must be 

carried on to keep combat troops in the field. 

PROGRAM 6: THE HUMAN SIDE OF BATTLE 

A presentation of the functions of the special staff officers 

concerned with the personnel aspects of combat operations, 
including treatment of prisoners of war and enemy civilians, 
as well as own troops. 

PROGRAM 7: THE CONDUCT OF THE ATTACK 

An exemplification of the principles of war as applied to the 
battle in progress . A description of the procedures for meeting 
emergencies and for modifying the original plan of attack in 
adjusting to unexpected enemy movements. 

PROGRAM 8: TEAMWORK IN THE ATTACK 

A review of all factors which must be taken into consideration 

in winning a battle; summaries of the high points of all the 
preceding instructional sessions with special emphasis on 
influence of leadership and necessity for teamwork. 



C. SCRIPT WRITING, DIRECTION, AND PRODUCTION OF PROGRAMS 

The basic plans for the programs were prepared by members of the faculty of 
the Army Command and General Staff College at Fort Leavenworth. These lessons 
conformed, in many respects, to demonstration lessons involving map problems 
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which had been presented to students at the Command and General Staff College. At 
the Army Signal Corps Photographic Center a team of two writers, a producer, and 
a director was assigned to each of the eight programs. These Hearns worked in 
close cooperation with a group of officers from the Command and General Staff 
College or from the various Arms and Services whose activities were represented 
in the programs. The full facilities of the Army Signal Corps Photographic Center 
were utilized in preparing and producing the lessons. Crews of artists prepared 
the maps and other graphics, special film sequences were photographed, and stock 
footage was located for other scenes. Large numbers of scene painters, stage 
carpenters, and electricians were also used in the productions. Inelaborateness of 
production detail, these programs were comparable to major network television 

shows • 

The s taf f of the Television Evaluation Project had no opportunity to influence 
the content of any program or the treatment accorded any topic. Because©! a close 
time schedule, it was necessary for the Evaluation Group to work with preliminary 
versions of the scripts in preparing test questions. Later changes in the scripts 
resulted in the deletion of some program content about which test items had been 

prepared. 

In early discussions regarding the programs, the Evaluation Group had requested 
that specific objectives be stated for each section of each program in order that 
attainment of these specific objectives might be tested. Despite repeated requests 
for these specific objectives, no explicit statement of them was obtained for any 
program. Hence it was necessary for the Television Evaluation Project staff to 
deduce from the scripts what the implicit objectives of each program were conceived 
to be by the team of script writers. 

Professional actors presented the narrative sequences and played the roles of 
the Division Commander and his staff. The only military personnel appearing on 
the programs were high-ranking officers who presented speeches either at the 
beginning or the close of the session. These speeches had a public relations objective 
and were not closely related to the instructional content of the program. 



D. TESTS 

Test items were developed by the Evaluation Group from the scripts as they 
were being prepared. Final scripts could not be used ? ir the preparation of questions, 
as script revisions were practically continuous up until the hour of the program, 
whereas test questions had to be prepared more than three weeks in advance of the 
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program to permit printing test booklets and distributing them to the widely scattered 
viewing groups. 

Multiple choice teat items providing four alternative responses were used. 
Samples of these items are shown below. The test questions were reviewed by 
faculty members of the Army Command and General Staff College, and correct 
answers agreeaupon for the construction of scoring keys. There was no opportunity 
to tryout items before they were printed in the test booklets, and the appropriateness 
of the difficulty level of the questions was dependent on the expert judgment of Project 
consultants. The test given before a training session typically consisted of two 
groups of items, as illustrated in Figure 2. The first group related to topics that, 
according to the latest available script, were to be treated in that training session. 
It measured knowledge of the topics prior to the television session. The second 
group of items tested knowledge of content of previous programs which had been 
tested from one to six weeks earlier. These items were included so that measures 

The final test for a given program con- 
sisted of a repetition of those items from 
the pretest booklet which related to the 
instructional content of that program, sup- 
plemented by at least an equal number of 
items also relating to the same program 
but which had not appeared earlier in .any 
test booklet. These additional questions 
were included in each final test so that, as 
the series of programs progressed, train- 
ees would not tend to concentrate solely 
on those portions of the lessons which 
would provide correct' answers to the 
questions asked in the pretests. 



of retention could be obtained. 




Samples of Test Items* 

PROGRAM 2: ADVANCE TO CONTACT 

In this situation, ground reconnaissance and security are provided the 20th 
Infantry Division by 

(A) the 201st Armored Cavalry, (B) the 201st Armored Cavalry and the 
205th Reconnaissance Company , (C) the regimental combat teams, (D) the 
20th Tank Battalion. 

* Correct answers are underlined on these samples only. 
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PROGRAM 2: ADVANCE TO CONTACT 

Use of regimental combat teams permits the Division Commander to 

(A) decentralize control to subordinate commanders . (B) spread out his 
forces over the entire area, (C) meet all aggressor threats, (D) pass 
responsibility for accomplishment of the mission to the combat team com- 
manders. 

PROGRAM 3: THE DIVISION COMMANDER'S DECISION 

In average terrain and against normal resistance, the most frequent formation 
of an Infantry Regiment in attack is 

(A) one Battalion in assault and two Battalions in reserve, (B) two Battalions 
in assault and one Battalion in reserve , (C) two Battalions plus in assault 
and one Battalion minus in reserve, (D) three Battalions in assault. 

PROGRAM 4: PLANNING THE ATTACK 

Blind spots in the anti-aircraft artillery radar screen are covered by 

(A) warnings from Corps, (B) visual observer teams . (C) reports from Recon- 
naissance Units, (D) fire of Infantry Units. 

PROGRAM 5: THE LOGISTICAL PLAN 
Supply economy is a function of 

(A) Division Quartermaster, (B) Division G-4, (C) Command . (D) Division 
Ordnance Officer. 

PROGRAM 6: THE HUMAN SIDE OF BATTLE 

Direct responsibility for safeguarding prisoners of war lies with 

(A) G-l, (B) the Provost Marshal . (C) the Adjutant General, (D) G-4. 

PROGRAM 6: THE HUMAN SIDE OF BATTLE 

World War H experience indicates that during three days of combat, an Infantry 
Rifle Company’s losses from all causes may be expected to be 

(A) between 9 and 12 per cent, (B) between 12 and 17 per cent. (C) between 
17 and 24 per cent, (D) more than 24 per cent. 

For the first program no script was available in time to permit preparation of 
tests, but a questionnaire to determine the attitude of reservists toward television 
instruction was administered at the conclusion of this program. 

In the tests for Programs 2 to 8 a total of 234 test items was used. One hundred 
of these items were used for repeated testing, and each one appeared both in the 
pretest and in the final test for some one of the programs. Thirty- seven of these 100 
items were also used for measuring retention and these items were presented for a 
third time after an interval of from one to six weeks. 
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The number of questions presented both before and after a particular lesson 
varied from 12 to 22 except for the seventh program which had only six items in the 
pretest. The seventh program consisted largely of illustrative and descriptive 
material which was specific to the conduct of this particular attack - material 

which could not be pretested fairly in any detail. 

In the final test booklets for Programs 2 to 7, participants were invited to make 

free comments on the acceptability of the program. The final test booklet for the 
eighth program included a rating scale consisting of 21 items, on which reservists 
were to record their attitudes toward various aspects of television instruction. 



E. TEST ADMINISTRATION 

The tests with directions for administering and for return shipment, were 
distributed through Army channels to Military District Commanders by Army Field 
Forces personnel. All tests were administered by the officer in charge of the 
various training groups. Thirty minutes immediately before each television session 
was allowed for the pretest and the same amount of time was allowed immediately 
after each session for the final test. These time limits permitted completion of the 

tests by all trainees. 



F. SCORING PROCEDURE 

All completed pairs of pretes.sand final tests were scored by means of stencils 
constructed from the scoringkeys preparedwith the help of advisors from the Army 
Command and General Staff College. The score on each test was the number of cor- 
rect answers marked. To obtain measures of retention, response sequences of 
individuals were tabulated. These sequences were analyzed to show whether the 
individual marked a given item correctly or incorrectly on the pretest, correc y 
or incorrectly on the final test, and correctly or incorrectly on the delayed reca.l 
test. This type of analysis is quite time-consuming, and consequently it was con 
sidered desirable to use a representative sample of the total population tested, 
rather than to analyze response sequences for all trainees tested. 



G. SUBJECTS 

The trainees participating in this study constitute a sample of Army Field Force 
Reservists ranging from private to colonel, selected only on fortuitous factors. To 
be included in the sample for a given week, a reservist needed to: 
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a. belong to a training organization operating within television reception 
range of one of the stations which agreed to transmitthe series ofpro- 

grams . 

b. belong to a training organization that was successful in securing on a 
few days' notice one or more television receivers. 

c. be in attendance during a particular telecast. 

d. complete both pretest and final test. 

For a given program, the sample was composed in part by individuals who had 
participated in all of the previous television sessions, and in part by reservists who 
had participated in varying numbers of the earlier television lessons. Officers were 
considerably more regular in attendance at consecutive training sessions than were 
enlisted men. However, the samples for the various weeks showed similar distribu- 
tions of cases drawn from the various personnel categories. 

More than 3000 reservists assembled for training each week in 160 viewing 
groups .♦ Tests were retained for scoring only if a complete pretest could be matched 
with a complete final test for the same individual. As shownin Table 1, from 2300 
to 2600 complete sets of test papers were available for Programs 2 through 8, with 

TABLE 1 

COMPLETED SETS OF TESTS GIVEN BEFORE AND AFTER INSTRUCTION 
FOR GROUPS CLASSIFIED BY RANK AND BRANCH OF SERVICE 



Category 


Test 2 


Test 3 


Field Grade 


Arms 


387 


350 




Services 


411 


363 


Comparty Grade 


Arms 


626 


653 




Services 


567 


529 


Sergeants 


Arms 


135 


139 




Services 


154 


161 


Other Enlisted 


Arms 


177 


190 




Services 


150 


155 


Total Group 




2607 


2540 



Test 4 


Test 5 


Test 6 


Test 7 


Test 8 


247 


371 


330 


352 


342 


244 


395 


379 


375 


355 


428 


629 


562 


604 


561 


378 


546 


509 


575 


527 


88 


130 


117 


121 


116 


88 


160 


111 


147 


139 


145 


181 


164 


180 


158 


116 


182 


140 


149 


144 


1734 


2594 


2312 


2503 


2342 



• Procrams were viewed by many military personnel in addition to those in the 160 television classrooms. Ap- 
oroximatelv 25 000 Army Field Force reservists within the television area were surveyed and it was found that 
more than 10 per cent viewed one or more of the programs, alone or in informal groups. Fourteen per cent of 14,400 
National Guardsmen surveyed reported that they had viewed one or more of the Command Post lessons. 
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the exception of Program 4, where only 1734 usable pairs of tests were obtained 
because test distribution for that program was delayed by a rail embargo. 



RESULTS 



A. DID THE RESERVISTS LEARN FROM THE TELEVISION INSTRUCTION? 

The total group of reservists showed a mean gain for each program in the 
score earned on the pretest items repeated after instruction. These mean gains 
were all statistically significant at the one per cent confidence level. Mean scores 
for the total group on pretest items* before and after each of the programs are 
reported in Table 2. This table also reports standard deviations of the distributions 
of scores. 

Although the mean gains were small numerically, the average improvement, 
considering all the programs, approximated one standard deviation of the distribu- 
tion of pretest scores before instruction. In other words, the average viewer improved 
his score as a result of the -instruction from a point where he exceeded 50 per cent 
of the group to a point where he exceeded 84 per cent of the scores earned before 
instruction. The fifth and sixth programs resulted in the largest mean increases 
in score relative to the variability of the group . For these programs the hypothetical 
average viewer increased his score after instruction from the initial 50 per cent 
point to a point where he exceeded 91 per cent of the scores earned before the lesson. 



B. DO ALL RANKS iH HETEROGENEOUS GROUPS BENEFIT FROM A SINGLE LEVEL OF INSTRUCTION? 

To determine whether the television programs were equally effective for all 
ranks, means and standard deviations were computed for scores on pretests and for 
scores on test items repeated after instruction, for each of eight personnel categories. 
These eight categories were composed of four classifications on the basis of rank, 
with each of these divided into ,, arms M and "services," as shown below: 



Rank Classification 



"Field Grade" 
"Company Grade" 
"Sergeants" 
"Other Enlisted" 



colonels, lieutenant colonels, majors 
captains, first lieutenants, second lieutenants 
all ranks of sergeants 
corporals, privates first class, privates 
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TABLE 2 



MEAN SCORES ON TESTS GIVEN BEFORE AND AFTER INSTRUCTION 
FOR GROUPS CLASSIFIED BY RANK AND BRANCH OF SERVICE 



PROGRAM 2 PROGRAM 3 









Pretest 


Repeated 

Items 




Pretest 


Repeated 

Items 


Category 




N 


M 


St) 


M 


SD 


N 


M 


SD 


0 

M 


SD 




Arms 


387 


8.4 


1.8 


9.3 


1.5 


350 


12.9 


2.0 


15.4 


1.6 


Field Grade 


Services 


411 


7 5 


1.8 


8.9 


1.5 


363 


11.8 


2.5 


14.7 


2.0 




Arms 


626 


7.7 


1.7 


9.1 


1.4 


653 


12.3 


2.0 


14.9 


1.7 


Company Grade 


Services 


567 


7.3 


1.8 


8.8 


1.3 


529 


11.6 


2.3 


14.5 


1.9 




Arms 


135 


6.8 


1.8 


8.2 


1.8 


139 


10.4 


2.4 


13.4 


2.5 


Sergeants 


Services 


154 


6.3 


1.8 


8.0 


1.8 


161 


9.9 


2.5 


12.5 


2.6 




Arms 


177 


5.8 


1.8 


7.4 


2.0 


190 


9.4 


2.7 


11.9 


2.9 


Otlier Enlisted 


Services 


150 


5.8 


1.7 


7.6 


1.9 


155 


8.8 


2.9 


11.8 


3.0 


Total Group 




2607 


7.3 


1.9 


8.7 


1.6 


2540 


11.5 


2.6 


14.2 


2.4 








PROGRAM 4 






PROGRAM 


5 










Pretest 


Repeated 

Items 




Pretest 


Repeated 

Items 


Category 


N 


M 


SD 


M 


SD 


N 


M 


SD 


M 


SD 




Arms 


247 


13.2 


2.8 


15.8 


2.4 


371 


3.5 


2.4 


11.9 


1.8 


Field Grade 


Services 


244 


11.2 


2.9 


14.6 


2.8 


395 


8.0 


2.5 


11.4 


2.3 




Arms 


428 


12.6 


2.7 


15.5 


2.3 


629 


7.6 


2.2 


11.5 


1.9 


Company Grade 


Services 


378 


11.2 


2.9 


14.4 


2.5 


546 


7.3 


2.5 


11.2 


2.1 


Sergeants 


Arms 


88 


10.2 


3.2 


13.6 


3.0 


130 


5.0 


2.1 


9.2 


2.4 


Services 


88 


9.1 


2.9 


11.9 


3.2 


160 


5.2 


2.4 


8.8 


2.5 




Arms 


145 


8.7 


3.2 


11.5 


3.6 


181 


4.8 


1.9 


8.1 


2.6 


Other Enlisted 


Services 


US 


8.8 


2.7 


11.5 


3.6 


182 


4.4 


2.0 


7.8 


2.7 


Total Group 




1734 


11.3 


3.2 


14.3 


3.1 


2594 


7.0 


2.7 


10.7 


2.6 
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TABLE 2 (Con*.) 



PROGRAM \ 

Repeated 

Pretest Items 



Category 




N 


M 


SD 


M 


SD 




Arms 


330 


8.1 


1.9 


11.5 


1.7 


Field G^ade 


Services 


379 


7.9 


1.8 


11.0 


2.0 




Arms 


562 


8.0 


1.9 


11.3 


1.8 


Company Grade 


Services 


509 


7.7 


1.9 


10.7 


1.8 




Arms 


117 


6.9 


1.9 


9.8 


2.1 


Sergeants 


Services 


111 


7.2 


2.1 


9.5 


2.5 




Arms 


164 


6.0 


2.1 


8.7 


2.3 


Other Enlisted 


Services 


140 


6.0 


1.9 


8.5 


2.4 


Total Group 




2312 


7.6 


2.0 


10.6 


2.2 



PROGRAM*? 

Repeated 
Pretest Items__ 



N 


M 


SD 


M 


SD 


352 


4.8 


1.1 


5.3 


0.7 


375 


4.5 


1.2 


5.1 


0.9 


604 


4.7 


1.2 


5.2 


0.9 


575 


4.4 


1.2 


5.1 


0.9 


121 


4.0 


1.3 


4.8 


1.0 


147 


3.6 


1.3 


4.7 


1.1 


180 


3.5 


1.5 


4.3 


1.3 


149 


3.4 


1.5 


4.5 


1.3 


2503 


4.4 


1.3 


5.0 


1.0 



PROGRAM 8 

Repeated 
Pretest Items 



ratpcrorv 




N 


M 


SD 


M 


SD 




Arms 


342 


24.3 


2.8 


24.6 


2.9 


Field Grade 


Services 


355 


23.1 


3.6 


23.8 


3.3 




Arms 


561 


23.5 


3.0 


24.3 


2.7 


Company Grade 


Services 


527 


22.6 


3,3 


23.5 


2.8 




Arms 


116 


20.5 


3.8 


21.8 


3.1 


Sergeants 


Services 


139 


19.8 


4.5 


21.3 


4.0 




Arms 


158 


18.0 


4.8 


19.3 


4.7 


Other Enlisted 


Services 


144 


18.4 


4.3 


18.8 


4.9 


Total Group 




2342 


22.3 


4.0 


23.1 


3.7 






Arms 



Services 






Infantry 
Cavalry 
Field Artillery 



Coast Artillery 
Chemical Warfare Service 
Corps of Engineers 
Signal Corps 



Adjutant General’s Department 

Judge Advocate General’s Department 

Quartermaster Corps 

Ordnance Department 

Medical Department 

Corps of Chaplains 

Corps of Military Police 

Finance Department 

Transportation Corps 

Inspector General’s Department 

Women’s Army Corps 



Means and standard deviations of scores on pretests and on pretests repeated 
alter instruction are given for each of the eight personnel categories in Table 2. 
Figure 3 presents in graphic form the mean scores for the eight personnel cate- 
gories on the tests given before and after the sixth program, and is representative 
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of the findings for the other programs. Before instruction, the mean scores lor 
reservists in the Field Grade, Company Grade, Sergeants, and Other Enlisted 
categories form a rather consistently descending scale for each program. Each 
grade of personnel in the Arms earned, on the average, a score which was slightly, 
but rather regularly, superior to the average score earned by the corresponding 
grade in the Services. These two gradients persisted in meanscoreson tests repeated 
after instruction. 

The mean gains made by each of the eight personnel categories for each of the 
seven programs were all statistically significant. For a given program the dif- 
ferences in gains made by the various ranks were relatively small. From program 
to program there was no regularity ; in the relative size of the gains made by the 
different personnel categories. Table 3 reports mean gains earned by categories 
grouped in a number of ways. 

C. DO TRAINEES RETAIN WHAT THEY LEARN FROM TELEVISION INSTRUCTION? 

Of the 100 items given immediately before and immediately after the program 
to which they pertained, 37 items were presented for a third time in a pretest 
booklet after an interval of from one to six weeks following the original testing. To 
determine the degree of retention over each interval, it was necessary to study the 
responses of individuals before instruction, after instruction and after the delay 
interval. A stratified sampling of the total tested population was drawn so that it 
would be representative of the entire participating reserve group in terms of grade, 
branch and Army area. In analyzing the response sequences for a single item, 
responses of all individuals in the stratified sample who had marked the item on 
the three occasions were tabulated. Hence, for that item, the per cent of correct 
responses before instruction, the per cent of correct responses immediately after 
instruction, and the per cent of correct responses after the one- to six- week interval 
could be calculated. For each of the 37 items, an average of approximately 600 
complete sequences was obtained for officers, and approximately 350 complete 
sequences for enlisted men. Table 4 presents the increase in percentage of correct 
responses resulting from instruction, and for each of the three intervals, the per- 
centage of this gain which was retained. Figures 4 and 5 present this same material 
in graphic form. JFor both officers and enlisted men, the degree of retention over 
periods of one week, two to four weeks, and five to six weeks was very high. Officers 
retained considerably more of the material learned as a result of instruction than 
did the enlisted men, either because of broader backgrounds, or because the instruc- 
tion resulted in re- learning of partially forgotten material. 
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TABLE 3 

MEAN GAINS > T TESTS GIVEN BEFORE AND AFTER INSTRUCTION FOR 
GROUPS CLASSIFIED BY RANK AND BRANCH OF SERVICE 



Category 
Field Grade 

Company Grade 

Sergeants 

Other Enlisted 

Total Group 

Field Grade Offs. 
Co. Grade Offs. 
Sergeants 
Other Enlisted 

Officer 

Enlisted 

TOTAL 

Officer 

Enlisted 

Arms 

Services 





Test 2 


Arms 


0.9 


Services 


1.4 


Arms 


1.4 


Services 


1.5 


£rms 


1.4 


Services 


1.7 


Arms 


1.6 


Services 


1.8 




1.4 




1.2 




1.4 




1.6 




1.7 


Arms 


1.2 


Services 


1.6 


Arms 


1.5 


Services 


1.8 




1.3 




1.6 




1.2 




1.6 



Test 3 


Test 4 


2.5 


2.6 


2.9 


3.4 


2.6 


2.9 


2.9 


3.2 


3.0 


3.4 


2.6 


2.8 


2.5 


2.8 


3.0 


2.7 


2.7 


3.0 


2.7 


3.0 


2.7 


3.1 


2.8 


3.1 


2.8 


2.7 


2.6 


2.8 


2.9 


3.3 


2.7 


o 

• 

CO 


2.9 


2.6 



2.7 


3.1 


2.8 


2.9 


2.7 


2.9 


2.8 


3.2 



Test 5 


Test 6 


3.4 


3.4 


3.4 


3.1 


3.9 


3.3 


3.9 


3.0 


4.2 


2.9 


3.6 


2.3 


3.3 


2.7 


3.4 


2.5 


3.7 


3.0 


3.4 


3.2 


3.8 


3.1 


3.9 


2.7 


3.3 


2.6 


3.6 


3.3 


3.7 


3.1 


3.6 


2.8 


3.5 


2.4 



3.7 


3.2 


3.6 


2.7 


3.7 


3.2 


3.6 


2.9 



Test 7 


Test 8 


0.5 


0.3 


0.6 


0.7 


0,5 


0.8 


0.7 


0.9 


0.8 


1.3 


1.1 


1.5 


0.8 


1.3 


1.1 


0.4 


0.6 


0.8 


0.5 


0.5 


0.5 


0.8 


0.9 


1.3 


1.0 


0.9 


0.4 


0.6 


0.7 


0.8 


0.8 


1.3 


1.1 


0.9 



0.6 


0.7 


0.9 


1.1 


0.5 


0.7 


0.8 


0.8 
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TABLE 4 



AMOUNT OF LEARNED MATERIAL RETAINED BY OFFICERS AND ENLISTED MEN 
OVER INTERVALS OF ONE WEEK, TWO TO FOUR WEEKS, AND FIVE TO SIX 



No. of Items 
Repeated after 
Indicated 
Interval 



No. of Response 
Sequences 
Analyzed 



Increase in 
Percentage 
of Correct 
Responses 
after 

Instruction 



Interval 

between 

Instruction 

and 

Delayed 

Testing 



No. of 
Percentage 
Points Lost 
over 
Specified 
Interval 



Per Cent of 
Increase 
Retained 
over 

Specified 

Interval 







OFFICERS 






20 


11,554 


20.1 1 week 


1.1 


94.5 


9 


5,010 


29.3 2-4 weeks 


2.5 


91.5 


8 


5,316 


15.6 5-6 weeks 


2.3 


85.3 






ENLISTED 






20 


6,985 


17.5 1 week 


3.4 


80.6 


9 


2,729 


29.9 2-4 weeks 


6.4 


78.6 


8 


2,899 


18.7 5-6 weeks 


6.5 


65.2 



The data are rather conclusive In indicating that not on^dldthereserWste 
lsai .„ from the television instruction, but that they retained substantial amounts of 
the material over periods of three to six weeks. 



D. WERE THE TRAINING PROGRAMS FAVORABLY RECEIVED BY THE RESERVISTS? 

The rating scale and questionnaire marked by reservists after tt.e eighth pro- 
gram showed that the series of television instructional sessions hau ueen v iy 
Cably received by the great majority of those who view*, 
pleted rating forms were availablefor 1,079 officers and 619 enlisted men. Eig ty 



PER CENT OF NEWLY LEARNED MATERIAL 




AFTER INSTRUCTION WEEKS AFTER INSTRUCTION WEEKS AFTER INSTRUCTION 

Figure 4. Percent of Learned Material Retained by Officers 




Figure 5. Percent of Learned Material Retained by Enlisted Men 
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four per cent of those- marking the rating form indicated they had viewed six 
or more of the programs and 39 per cent stated they had viewed the entire 

series. 

The questions asked, the response alternatives provided, and the percentage of 
officers and of enlisted men indicating each of the responses are presented graphically 

on the following pages. 

There was substantial agreement between the views expressed on the rating 
scale given after the first program and those indicated on the rating scale covering 
the whole series of programs and marked immediately after the eighth program. 
The only important change in attitude was in the direction of more favorable rating 

of the instructors for the later programs. 

In addition to the rating scales presented at the close of the first and eighth 
programs, participants in each television lesson were invited to write, in a space 
provided in the post-test booklets, any comments they cared to make regarding the 
program just viewed. These comments were found to parallel rather closely the 
views expressed in the rating scales. The principal supplementary information 
supplied by the free comments was the opinion, expres^d by about 10 per cent of 
the officers and 5 per cent of the enlisted men, that the long introductory or closing 
speeches by high-ranking officers were extraneous or poorly executed. The free 
comments on the first five programs included frequent complaints that the visual 
aids were poorly chosen or poorly presented. 



1. Compared to my other Reserve training periods, the television programs have been 





q. 

As 


Very interesting 


69.2 




41.4 


interesting 


26.3 




40.4 


Average 


3.5 




15.0 


Dull 


0.6 




1.6 


Very dull 


0.1 




0.8 





f~l Officers 
■ Enlisted Men 
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2. Compared to a typical Reserve training session, the television programs have been 



Very difficult 



% 



o 

u 



1.7 0 
1.5 i 



Per Cent 



1 5 



SO 

_l 



73 



100 

—I 



Difficult 



Average 



14.8 C 

12.8 I 



43.3 C 
46.9 



Easy 



Very Easy 




10.3 cm 

6.1 ■ 



□ Officers 
■ Enlisted Men 



3. Compared to the average training films which I have seen within the past two years, the 
television programs were 



More Instructive 



% 



o 



72.4 : 
56.2 



25 

I 



Per Cent 
50 



73 

i_— 



100 



About the same 



19.3 

2o.o 



□ 



Less Instructive 



7.0 □ 
15.0 1 
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4. During atypical service training day, how many hoursdo you feel you could watch television? 



Less than 1 hour 

1 hour but not 

2 hours 

2 hours but not 

3 hours 

3 hours but not 

4 hours 

4 hours or more 



% 

0 29 

1 

l.i i 

3.4 ■ 

20.7 l — 3 

32.0 ■■■■ 

48.4 I — 

36.8 

12.8 CIZ) 

16.5 I _3 
14.7 



79 IOO 

I 



O Officers 
g&B Enlisted Men 



Per Cent 

90 



5. 



During a typical service training day, how long an interval do you feel would be necessary 
between television programs? 



% 



0 

1 



Per Cent 

29 90 

=t== 1 



79 *00 
_i 1 



Less than 1 hour 



22.2 I H3 

38.6 



1 hour but not 
2 hours 



25.9 C 
20.7 I 



2 hours but not 
3 hours 



19.3 [ 

12.3 I 



3 hours but less 
than 1 day 



24.2 [ 
12.9 I 



1 day or more 



5.4 □ 

7.3 m 



O 



20 



^ "V 



6. In general did you feel that the subject matter was presented 



% 



Per Cent 



0 25 50 75 

1 1 ' L 



100 

I 



Too rapidly 


27.0 




31.4 


About right 


68.9 




63.5 


Too slowly 


2.4 




4.0 




7. Did you feel that the subject matter was presented 

% 



In too much detail 



o 

L. 



25 

I 




Per cent 



50 

i- 



75 



too 

_i 



About right 



75.0 C 
60.3 



In too little detail 



14.4 

13.3 



8. In general were the programs 



% 



o 



Too long 


8.7 


cm 




16.6 




About right 


88.1 


nr 




77.4 




Too short 


2.8 


□ 




4.5 


■ 



25 

i 



Per Cent 



so 

I 



75 
I - 



100 
I 



... 9 



* 



o 

ERIC 

& 
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9. Do you think too many topics were covered in the typical program? 



Yes 




No 



72.4 l t 

55.6 



O Officers 



Enlisted Men 



10. Ini the film sequences were the essential elements of the motion picture film used — 

% Per Cent 

0 25 80 75 K;0 

1 I * J- » 



a. Readily visible? 

Yes 74.1 

76.1 

No 24.3 i 1 

20.2 




b. Readily understandable? 

Yes 



No 



80:9 I ~ — 1 

75.1 

15.3 i ~i 
13.7 BM 
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11. In general, did the motion picture film a<?--3 to the understanding of the material? 



Yes 



% Per Cent 

0 25 5G 75 100 

1 I I I 1 

90.6 I - ] 

87.5 



No 



7.4 CU 

10.8 mm 



n Officers 
■i Enlisted Men 



12. Where maps and charts were used, were the essential elements readily visible and under- 
standable? 



% 



Per Cent 



Yes 



0 25 50 

1 I I 



34 .4 1 3 _ D 

46.9 



75 100 

-I ( 



No 



63.8 I 3 

51.6 IHHHH 



If not, why? 

Printing too small 
on some 



45.7 1 I 

33.8 



Insufficient time to read 
charts or maps 



36.2 1 =□ 

27.3 



Parts of the maps or 

charts were off the screen 



14.1 CU 



Other. Specify 



23.8 I ~ 1 
13.7 ■■■ 
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13 If you felt there were any instances where models, maps, or charts rather than the presence 
‘ of the lecturer on the screen would have been more helpful in promoting understanding of the 

material, did this occur 



% 



0 

L 



Per Cent 



25 



50 75 

_l L 



100 

_j 



Very frequently ? 


2.1 




3.7 


Frequently ? 


19.6 




28.0 


Seldom ? 


70.1 




58.8 


Never ? 


5.7 




5.2 
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14. On what size television screen did you usually view the programs? 



10” or smaller 


8.9 




10.3 


12” or 12-1/2" 


24.6 

26.7 


15” or 16" 


34.0 

25.5 




28.7 


19", 20", or projection 


33.0 



Per Cent 
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75 

-J— 



100 

1 
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15. Was the television screen you usually viewed 

% 



Too big 


1.1 




1.5 


Adequate 


81.1 




81.4 


Too small 


16.8 




15.0 



Per Cent 

0 25 50 75 100 
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16. Did you feel that the illumination of the screen was, in general 



% 



Too bright 


5.2 




6.9 


About right 


56.6 




56.7 


Not bright enough’ 


11.7 




11.6 


Flickering 


32.6 




28.3 



Per Cent 
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17. In general, did you find that the room illumination interfered with your viewing of the 
screen? 



Yes 



% Per Cent 

0 25 50 75 100 

1 I J- 1 1 

i4.i r ~ ~) 

17.8 



No 



84.6 I .... — - = 3 

80.2 
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18. Were you usually able to hear the programs adequately? 



Per Cent 



Yes 



% 



o 

L_ 



25 

i — 



50 



75 
l — 



EDO 



93.3 Cl 
91.0 « 



No 



5.8 EH 




If not, why? 



Too far away from 


0.4 
0.8 1 


source of sound 


Sound too loud 


0.4 

0.2 


Not loud enough 


0.3 
1.1 1 


Garbled and distorted 


3.0 □ 
4.4 ■ 


Other. Specify 


2.1 D 

1.1 1 
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9. Give a composite (average or over-all) rating of the principal instructors in the programs 
with respect to the four following characteristics: 



Per Cen 



a. Personality 



Very good 


63.6 

50.3 


| 




Good 


31.9 


| 


41.5 




Fair 


3.0 


□ 




5.5 




Poor 


0.1 






0.6 
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75 
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100 
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b. Speech characteristics 

Very good 58.9 I 

52.0 I 

Good 26.2 C 

38.5 I 




Fair 



3.3 □ 



0.3 

0.6 



Poor 



19. (cont) 





% Per Cent 


c. Command of subject matter 


O 2S SO 7 ? , ° 1 ° 

1 L 1 


Very good 


72.8 — 1 

55.1 


Good 


23.6 r — 1 


Fair 


2.2 □ 


Poor 


Q 2 □ Officers 

q 2 Hi Enlisted Men 



d. Ability to explain things clearly 


0 25 SO TS 100 

L 1 1 1 


Very good 


65.8 I — ' 

51.9 


Good 


30.0 r~ . J 

36.2 


Fair 


2.8 Q 


Poor 


0.5 

0.5 
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20. Given the opportunity to receive similar instruction either in the classroom or via television, 
which would you choose? 

% Per Cent 

O 25 50 79 100 

, | I I ' 

Classroom 20.0 1 1 

31.2 

Television 75.1 L.. — — ^ 

62.9 

□ Officers 
■I Enlisted Men 



21. What over- all rating would you assign the series of programs? 



Excellent 



% 



Per Cent 



o 
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25 
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50 



75 

_ 1 _ 
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25.7 I 



Good 



Fair 
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E. COMMENTS OF THE TRAINING EXPERTS 



The programs were viewed and commented upon by twenty civilian and military 
training experts. They were almost unanimous in expressing the belief that there 
was steady and significant improvement in the quality of the programs as the series 
progressed. The following statements summarize the most significant observations 

of the training experts: 

1. Changing emphasis in later programs from dramatics to illustrated 
narration improved the instructional quality of the lessons. 

2. Material was well presented in genera!, but content was more suitable 
for higher ranking officers. 

3. Too many topics were covered in most of the programs. 

4. The purpose of the instruction was not clearly defined. The over-all 
or general instruction was mixed too much with the specific, and the 
program became an orientation course and a technical course com- 
bined. 

5. The introductory and closing speeches by high-ranking officers were, 
with few exceptions, too long, quite boring, poorly reproduced, and 
detracted from the instructional value of the programs. Public rela- 
tions objectives of these speeches appeared to be mixed with troop- 
incentive objectives and neither purpose was effectively realized. 
Future training programs need to find better techniques for exploiting 
the prestige of high-ranking officers. 



F. WHAT EFFECT DOES THE DEGREE OF EXPLICITNESS OF TREATMENT OF TOPECS HAVE UPON THE 
LEARNING OF THE TOPICS? 

Kinescope recordings of Programs 2 to 8 were used to correct ’’final" scripts 
so that they corresponded exactly with the procedure actually followed in the tele- 
vision instructional sessions. As earlier stated, there was a total of 100 items 
included in the tests for Programs 2 to 8, which had been presented immediately 
before and immediately after a program in which preliminary scripts had indicated 
that the topics upon which items were based would be treated instructionally. The 
corrected scripts and the kinescope recordings were then used to determine the 
degree of explicitness of the instructional treatment accorded each of the topics 
underlying the 100 test items. 



30 



o 



The instructional treatment of a topic was rated as "explicit” if the meaning was 
brought out definitely in words, or plainly expressed either by graphics or by a 
dramatic sequence. A treatment was classified as "sketchy" if the program content 
relating to a topic was scattered, diffuse, or indirect, or which required one or 
more inferences on the part of the viewer to extract implicit meanings. A topic 
was considered to be "not covered" if analysis of the script and viewing of the 
kinescope recording failed to reveal any pertinent treatment of the topic. 

Two staff members, operating independently, assigned ratings of "explicit," 
"sketchy,” or "not covered” to the treatment given each of the 100 test items. If 
the two raters disagreed on the coverage of any item, the portions of the kinescope 
recording and the script dealing with the item in question were reviewed a third 
time by an additional staff member. After discussing the item, its treatment in the 
scriDt, and the presentation on the screen, this group of judges arrived at a decision 
concerning the degree of explicitness of treatment. There was, in general, close 
agreement by the raters . A total of 67 items were found to have been explicitly 
treated, 14 had been given a sketchy treatment, and 19 items were not directly 
covered in the television presentation. 

A stratified sample of officers was selected in such a way that the proportion of 
each grade and branch in each Army area in the sample corresponded to the per- 
centage of that grade and branch in each Army area contained in the total population 
tested. A stratified sample of enlisted men was similarly formed. The size of 
these samples was set high enough so that for each item a minimum of 250 response 
sequences (i.e., the responses of the same individuals before and after instruction) 
for officers and a like number of response sequences for enlisted men were available 
for tabulation. On the average, 350 response sequences were secured for officers 
and 275 response sequences for enlisted men. For these stratified samples, mean 
percentages of correct responses before and after instruction on each of the 67 
explicitly treated items were determined. Similar determinations were made for 
the 14 sketchily treated items and the 19 items not covered in the television sessions. 

Data for all the items included within a given treatment category were com- 
bined. Table 5 reports the obtained combined percentage of correct response 
before and after instruction, and Figure 6 presents the same data graphically. On 
the average, large and significant gains were made on those items which had been 
ejcplicitly treated in the television sessions. Items sketchily treated or not directly 
covered both showed small mean gains. It is possible that the slight gains made 
on items for which no direct treatment could be found, resulted from learning or 
reinforcement of related material, or from reduction of the likelihood of one of the 
wrong alternatives being selected. 
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table 5 

PERCENTAGES OF CORRECT RESPONSES FOR REPEATED ITEMS GROUPED BY 
DEGREE OF EXPLICITNESS OF TREATMENT 



Degree of 
Explicitness 


No. of 
Items 


Group 


No. of 
Response 
Sequences 
Analyzed 


Percentages of Correct Responses 

Before Immediately 

Instruction After Instruction 


Gains in 
Percentage 
Correct 






Officers 


25213 


60.6 


82.6 


22.0 


Explicit 


67 


Enlisted 


19060 


52.9 


68.4 


15.5 






Total 


44273 


57.3 


76.5 


19.2 






Officers 


4601 


75.7 


76.7 


1.0 


Sketchy 


14 


Enlisted 


3744 


57.9 


61.4 


3.5 






Total 


8345 


67.7 


69.8 


2.1 



Not Covered 



19 



Officers 

Enlisted 

Total 



5675 

4945 

10620 



59.9 

50.2 

55.4 



61.8 

52.6 

57.5 



1.9 

2.4 

2.1 



EXPLICIT 
(67 ITEMS) 



SKETCI'Y 
(14 ITEMS) 



NOT COVERED 
(19 ITEMS) 




Fi 9 ... 6. Percentage* .( Correct Responses (o, Items Given Betor. und -her Instruction Grouped 

by Coverage in Program 
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Table 6 reports the percentage of items receiving each of the three degrees 
of explicitness of treatment which showed, after instruction, significant gains or 
losses, or non- significant gains or losses. Considering data from the off icer sample, 
it was found that 81 percent of the items given explicit treatment showed statistically 
significant increases in the percentage of correct responses following instruction; 
4 per cent of the explicitly treated items showed significant losses. For the officer 
group 29 per cent of the sketchily treated items showed significant gains, and 14 
per cent of them showed significant losses; 16 per cent of the items not treated 
showed significant gains, and 11 per cent of them showed significant losses in per 
cent of correct responses following instruction. A rather similar picture is shown 
by the data for the enlisted men. Seventy- nine per cent of the explicitly treated 
items showed significant gains, and only 4 per cent showed significant losses. For 
the sketchily treated items, 21 per cent showed significant gains, and 14 per cent 
significant losses. Twenty-six per cent of the items not covered showed significant 
gains, and 16 per cent of them showed significant losses. 

G. DOES. THE TYPE OF INSTRUCTIONAL TREATMENT GIVEN A TOPIC INFLUENCE THE AMOUNT 
OF LEARNING? 

Kinescope recordings of the eight one-hour programs were analyzed to deter- 
mine the relative amounts of program time devoted to each of five types of activities: 



TABLE 6 

PERCENTAGE OF ITEMS GIVEN EXPLICIT TREATMENT, SKETCHY TREATMENT, 
OR NO COVERAGE WHICH SHOWED AFTER INSTRUCTION SIGNIFICANT GAINS, 
NON-SIGNIFICANT GAINS, NON-SIGNIFICANT LOSSES, AND SIGNIFICANT LOSSES 



Percentage of Items Receiving Indicated 
Treatment which after Instruction Showed: 

Degree of No. of Significant Non- significant Non- significant Significant 

Explicitness Items Group Gain Gain Loss Loss 



Explicit 



Sketchy 



Not Covered 



Officers 


81 


11 


4 


4 


Enlisted 


79 


8 


9 


4 


Officers 


29 


21 


36 


14 


Enlisted 


21 


29 


36 


14 


Officers 


16 


47 


26 


11 


Enlisted 


26 


37 


21 


16 
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(a) opening, intermission and closing, (b) speeches oy ranking officers, (c) narration 
with or without film, (d) atmospheric, or ’’scene setting” film without narration, and 
(e) dramatized action. The percentages of each hour consumed by each of these 
activities are reported in Table 7. It is to be noted that as the series progressed, 
the amount of drama decreased and the amount of narration increased. 

Kinescope recordings and corrected scripts were also analyzed to determine 
the type of instructional activity which had been used to convey information about 
the topics tested by the 100 items discussed in the preceding section. When the 
subject matter was treated by actors in dramatic sequences, the type of treatment 
was classified as drama ; when an instructor presented information in the form of a 
lecture, the treatment was classified as narration . If film accompanying explanations 
by the narrator was judged to convey meaning by itself, or to amplify the meaning 
conveyed by the narrator, the treatment was termed narration with mea ning-conveying 
film. In this series of programs, the only film judged to convey meaning, as described 
above, happened to be animated film. Where narration was accompanied by film 
whose function was to ’’seta scene” rather than to supplement the meanings conveyed 
by the narration, the treatment was classed as narration with atmospheric film . A 
few of the topics were given both dramatic treatment and some type of narration, 
and this category has been labeled drama combin ed w ith some type of narrat ion. 
Percentages of correct responses before and after instruction for each of the five 
types of treatment are given in Table 8 anu Figure 7. 

TABLE 7 

PERCENTAGE OF PROGRAM TIME IN EACH SESSION 
DEVOTED TO SPECIFIED KINDS OF ACTIVITIES 



Program 


Opening, 

Intermission, 

Closing 


Speeches 


Narration 
With or 
Without 
Film 


Atmospheric 
Film Without 
Narration 


Drama 


1 


3.5% 


28.1% 


29.8% 


1.8% 


36.8% 


2 * 


11.0 


22.7 


34.8 


1Qo7 


20.8 


3 


11.3 


8.2 


25.1 


5.4 


50.0 


4 


6.2 


10.8 


38.0 


5.5 


39.5 


5 


8.4 


6.7 


47.2 


1.3 


36.5 


6 


4.4 


17.7 


41.5 


3.0 


33.3 


7 


4.6 


9.3 


52.5 


15.3 


18.3 


8 


5.3 


12.2 


58.2 


11.3 


12.9 
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TABLE 8 



PERCENTAGES OF CORRECT RESPONSES FOR REPEATED ITEMS 
GROUPED BY TYPE OF TREATMENT 



Type of 
Treatment 


No. of 
Items 


Group 


No. of 
Response 
Sequences 
Analyzed 


Percentages of Correct Responses 

Before Immediately 

Instruction After Instruction 


Gains in 
Percentage 
Correct 


Narration 


27 


Officers 


9845 


59.1 


80.9 


21.8 






Enlisted 


7538 


47.4 


69.2 


21.8 






Total 


17383 


54.0 


75.8 


21.8 


Narration with 


16 


Officers 


5165 


67.4 


82.4 


15.0 


Atmospheric Film 




Enlisted 


4285 


55.6 


68.2 


12.6 






Total 


9450 


62.0 


76.0 


14.0 


Narration with ) 




Officers 


1652 


52.4 


87.0 


34.6 


Meaning* Conveying 


5 


Enlisted 


1344 


36.0 


69.6 


33.6 


Film 


















Total 


2996 


45.0 


79.2 


34.2 


Drama 


24 


Officers 


9521 


66.3 


77.9 


11.6 






Enlisted 


7045 


50.7 


62.2 


11.5 






Total 


16566 


59.7 


71.2 


11.5 


Drama Combined 




Officers 


3631 


62.7 


90.6 


27.9 


with Some Type 


9 


Enlisted 


2592 


43.9 


72.6 


28.7 


of Narration 


















Total 


6223 


54.9 


83.1 


28.2 
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Figur. 7. Percentages of Corr.et Responses for Items Given Before and After Instruction Group- 

ed by Type of Treatment 
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All types of treatment produced an increase in knowledge. Narration by itself 
or in combination was found to be extremely effective. The largest percentage 
increase in knowledge was found for those items treated by narration accompanied 
by meaningful film. Dramatic treatment by itself brought about smaller improvement 
than any of the other types of treatment recognized. However, items given dramatic 
treatment and also treated by some form of narration showed large mean gains. 

Table 9 reports the percentage of items given each type of treatment which 
showed after instruction (a) significant gains, (b) non- significant gains, (c) non- 
significant losses, and (d) significant losses. All five of the items treated by narration 
with animated film showed statistically significant gains for both officer and enlisted 
personnel. Eight of the nine items treated by drama combined with some type of 
narration showed significant gains for both officers and enlisted men. On the 27 
items treated by narration alone, officers showed significant gains on 20, and 
enlisted men significant gains on 19. It is interesting to note that the lower per- 
centages of significant gains found for items treated by narration with atmospheric 
film or by drama are associated with small, bui rather disturbing percentages of 

TABLE 9 

PERCENTAGE OF ITEMS GIVEN EACH TYPE OF TREATMENT WHICH SHOWED, 
AFTER INSTRUCTION, SIGNIFICANT GAINS, NON- SIGNIFICANT GAINS, 

NON- SIGNIFICANT LOSSES, AND SIGNIFICANT LOSSES 









Percentage of Ite« * 


Receiving Indicated 










Treatment which 


* Instruction Showed: 




Type of 
Treatment 


No. of 
Items 


Group 


Significant 

Gain 


Non- significant 
Gain 


Non- significant Significant 
Loss Loss 




27 


Officers 


74 


18 


8 


0 


Narration 


Enlisted 


70 


18 


8 


4 


Narration with 
Atmospheric Film 


16 


Officers 

Enlisted 


63 

69 


19 

0 


12 

19 


6 

12 


Narration with 




Officers 


100 


0 


0 


0 


Meaning- Conveying 


5 


Enlisted 


100 


0 


0 


0 


Film 


24 


Officers 


58 


13 


13 


16 


Drama 


Enlisted 


50 


21 


21 


8 


Drama Combined 




Officers 


89 


11 


0 


0 


with Some Type 
of Narration 


9 


Enlisted 


89 


0 


11 


0 
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items for which significant losses were found. Considering both significant and 
non- significant losses for these two types of treatments, there is more than a 
suggestion that confusion was aroused by the treatment accorded some topics. 



H. EXAMPLES OF EFFECTIVE AND INEFFECTIVE INSTRUCTIONAL TREATMENTS 



Effective and ineffective instructional procedures show characteristic dif- 
ferences. Clear, direct handling of a topic can usually be distinguished at the script 
stage of production from one which is indirect, vague, or ambiguous. But to make 
the distinction, one must be actively concerned with meanings rather than with the 
development of a ’’show" designed to fill an hour of broadcast time. Script writers 
and producers, left to their own devices, tend to lose sight of the objectives of 
instruction. The psycho- educational specialist has an important role to play in the 
production of effective instructional sessions. 

As an example of what may be accomplished by good television instruction, the 
learning and retention records of several hundred officers and enlisted men for a 
single item are presented together with a description of the treatment accorded this 
topic. The following item (12P4) was included in the pretest for Program 4: 

Tactical Air Control Parties are used to* 

(A) guide aircraft to the proper targets , (B) coordinate Anti-Aircraft 
Automatic Weapons fire, (C) control the type of attack made by aircraft on 
Aggressor positions, (D) coordinate the flights of the Organic Division 
liaison aircraft. 

The function of Tactical Air Control Parties was explicitly treated in the fourth pro- 
gram. The type of treatment given the topic was narration with scene-setting or 
atmospheric film combined with drama. The identical test item was repeated in the 
final test following Program 4. After a four-week interval, the same test item was 
included in the pretest before the eighth, or review, session. The topic of Tactical 
Air Control Parties was explicitly treated in the review program, tnistime by drama 
combined with narration. At the close of the review program, the same test item 
was repeated for the fourth time. 

Records were available from stratified samples for 435 officers and 256 enlisted 
men who had taken all four of the tests in which the Tactical Air Control item ap- 
peared, and had viewed both Program 4 and Program 8. Table 10 and Figure 8 

• (A) is the correct alternative 
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TABLE 10 



PERCENTAGES OF CORRECT RESPONSES GIVEN TO TEST ITEM 25P4 
BEFORE INSTRUCTION, AFTFR INSTRUCTION, AND BEFORE AND AFTER 
A REVIEW SESSION PRESENTED FOUR WEEKS AFTER ORIGINAL INSTRUCTION 











Four Weeks after Instruction 




N 


Before 

Instruction 


Immediately 
After Instruction 


Before 

Review Session 


After 

Review Session 


Officers 


435 


32.2 


95.4 


95.4 


97.5 


Enlisted Men 


256 


21.9 


84.4 


85.9 


89.5 



show (a) that officers and enlisted men benefited greatly from the original instruc- 
tion, (b) that there was no forgetting over a four- week interval (this was not typical), 
and (c) that the review further increased the high percentage of correct answers. 

The total program time devoted to the treatment of Tactical Air Force Parties 
was two minutes and 55 seconds, of which 47 seconds (Scene 74) of narration dealt 
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Figure 8. Tho Effect of Explicit Instruction upon Frequency of Correct Responses to Test Item 25P4 
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specifically with the question asked in Item 25P4. The following excerpt from the 
television script shows the complete treatment given to the topic of Tactical Air 

Control Parties: 






SCENE 68: EXTERIOR - G-3 HEADQUARTERS 



MEDIUM SHOT ON THE OPEN DOOR °F ™J^3 0FFICE, TmOUGH WfflCH 

tor 

HELLO, AND ENTERS THE G-3 OFFICE. 

Racl^at^he* G-3 office there's anew visitor arriving, Major MiUer, G-3 Air 
He's thf officer upon whom Colonel Baker will depend for working out the 

details of tactical air support. 



SCENE 69: INTERIOR - G-3 HEADQUARTERS 



MEDIUM SHOT ON MAJOR MILLER AS ENTERS G-3 OFFICE AND WALKS 

TO G-3 TO DESK, THE CAMERA PANNING HIM. 

CLOSE SHOT ON THE ASSISTANT G-3 ENTERING NOTES ON OPERATIONS 
ORDER. 

COLONEL BAKER’S VOICE: 

Sit down, Barney - where have you been? 

fw"°geK E uT^«cal E air control party oriented on the situation. 

MEDIUM SHOT ON THE G-3 AND MAJOR MILLER 

sssrtasf ass 

from Army. 

MAJOR MILLER: 

I’m ready. 

S5JK&8SSS S&35SWS&B VS S " m CM ” T0 LOOK 

need air attacks. 

SEE MAP #6 FOR ABOVE. 

EXTREME CLOSE-UP ON MAP AS G-3’s HAND POINTS OUT G-3’s SHADOWED 
PROFILE IN LOWER LEFT HAND CORNER. 

COLONEL BAKER'S VOICE: 

Take a look at this hill, here. 
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SEE MAP #6 

HAND POINTS BENCH MARK 595 (HILL). JUST NORTH OF MARYSVILLE. 

According to G-2, it’s covered with trees and honey- combed with caves, and 
aggressor troops have been observed there. 

MEDIUM SHOT ON THE G-3 AND MAJOR MILLER FROM ANOTHER ANGLE. 
COLONEL BAKER: (CONTINUING) 

Well — if the aggressor retains a sizeable force there, it'll become a danger 
area to our troops coming in from here. (HE POINTS) 

MAJOR MILLER: 

I see well, I’ll request pre-planned missions for those two areas. Tactical 

Air Force will probably select fighters armed with rockets to attack that tank 
concentration. 

COLONEL BAKER: 

The hill looks more like an area target, and the aggressor troops there will 
probably hide out in those caves. 

MAJOR MILLER: 

That’ll mean a job for the tactical bombers. 

COLONEL BAKER: 

Now, when you request the pre-planned missions, tell them we'll probably need 
’’immediate support” missions in the area southeast of Junction City where the 
aggressor may bring in some reinforcements. 

(FADED OUT) 



SCENE 69: TERRAIN MODEL 

MEDIUM SHOT ON NARRATOR II. 

NARRATOR II: 

An "immediate support" or "on call" mission is used when it is reasonably sure 
that an air attack will be needed, but when the time and the target are uncertain. 

DISSOLVE: 



SCENE 70: FILM 

SHOT OF PLANE ZOOMING AT LOW ALTITUDE AND HIGH SPEED 



SCENE 71: SHOT OF PILOT 



SCENE 72: AIR SHOT OF GROUND MOVING SWIFTLY BELOW 



SCENE 73: TACP OFFICER ON RADIO. (THIS CAN BE FAKED BY A SHOT OF 
ANYONE ON RADIO) 
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SCENE 74: SHOT OF PILOT TAL.HNG ON RADIO 

NARRATOR II: (CONTINUING) _ , . 

When a pilot flies an aircraft at high speed and at lo\. altitudes, he may have 
trouble identifying ground targets. To avoid mistakes, he is told the location of 
our own ground troops, and is guided to targets by Tactical Air Control Parties. 
In this situation, one such party is with each infantry regiment and one is with 

the division headquarters. _ . . „ ... .. 

These parties are composed of Air Force Personnel. By being with the front 
line troops, and by talking with the pilot by radio, they can guide him to the 
proper target, referring to easily identifiable lendmarks. 

DISSOLVE: 



SCENE 75: INTERIOR G-3 HEADQUARTERS 

MEDIUM SHOT ON THE G-3 AND MAJOR MILLER AT THE DESK. 

MAJOR MILLER: ^ , . . . .. 

I*m sure we can count on the tactical air control parties. 

COLONEL BAKER: 

Then that does it. 

G^thos^requests in proper form and assign priorities to the targets before 
we forward them to Corps. 

MAJOR MILLER: 

Yes, sir! 



In contrast to the direct, explicit treatment accorded the subject of Tactical 
Air Control Parties, may be cited the treatment given the topic of ’’Priority of Fires." 
The sequence of scenes in which this topic was covered consumed two minutes and 
55 seconds, the same running time as for the Tactical Air Control Parties topic. 
The subject of "Priority of Fires" was treated in a dramatic sequence consisting 
principally of dialogue between two officers, the Commander of the 60th Infantry 
Regiment, and the Commanding Officer of the 79th Field Artillery Battalion. The 
test item which related to this topic was Item 12P4:* 

"The priority of fires to be provided by direct support for the battalions of an 

attacking Infantry Regiment is decided by the 

(A) Commanding Officer of the supporting Field Artillery Battalion 

(B) Commanding Officer of the Infan try Regiment 

(C) Commanding Officers of the Infantry Battalions 

(D) G-3 of the Division Artillery." 

- (B) is the correct alternative 



42 



o 



The following excerpt from the television script presents the entire treatment 
given to arranging Priority of Fires. It is to be noted that the introduction of the 
two officers by the narrator at the opening of Scene 32 does not explicitly identify 
either of the two officers as the Regimental Commander or as the Commanding 
Officer of the 79th Field Battalion. Because it is necessary for an observer to infer 
from the dialogue the role played by each of the officers, and because the discussion 
is quite diffuse, the treatment given to the topic of Priority of Fires was rated as 
"sketchy.” 



SCENE 32: INTERIOR - 60th INFANTRY CP: 

MEDIUM SHOT ON COLONEL JAKE WATSON AND LT. COL. KEN CARSON, 
AS THEY STAND OVER AN OVERLAY MAP #5 RESTING ON A TABLE. 

NARRATOR II: 

Later, the commanding general did approve the plan for the 2Q«- minute prepara- 
tion. It is now being worked out in more detail in the lower echelons of the 
division. 

THE CAMERA STARTS TO MOVE IN VERY SLOWLY TO THE TWO AT THE 
TABLE. 

Here, in |the 60th Infantry CP, the Regimental Commander, Colonel Watson, is 
discussing attack plans with Colonel Carson, Commanding Officer of the 79th 
FA Battalion. 

COLONEL CARSON: 

I just heard from Division Artillery a few minutes ago. They tell me I'm in 
direct support of your regiment. 

COLONEL WATSON: 

That's right. Let's get started. Have a seat. 

CARSON SEATS HIMSELF. 

I'll show you my tentative plan on this map. 

COLONEL CARSON: 

O.K. 

THE CAMERA SHOULD NOW BE ON AN EXTREME CLOSE-UP OF THE MAP. 

EXTREME CLOSE-UP ON THE MAP (INSERT CAMERA) 

(ALL POINTS ARE MARKED OUT IN OVERLAY 5 TO GO WITH MAP 5) THE 
PROFILED SHADOWS OF BOTH MEN ARE SEEN IN THE BOTTOM CORNERS 
OF THE MAP. 
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COLONEL WATSON’S VOICE: (CONTINUING) 

I plan to attack with battalions abreast. The 1st on the right and the 3rd in 
reserve. I’m also holding the 20th Tank Battalion in regimental reserve. The 
regimental tank company will be attached to the 1st battalion, except for one 
tank platoon, which will be attached to the 2nd battalion. 

COLONEL CARSON’S VOICE: 

This is the boundary between your battalions here. 

INDICATED IN OVERLAY 5 

COLONEL WATSON’S VOICE: . ... . .. .. 

Right! Now, the 1st battalion gets priority of fires. It’s lmtial objective is 
here. And, from here, the battalion will go on to Division Objective #1, right 

here. 

INDICATED IN OVERLAY 5 

MEDIUM SHOT ON COLONEL CARSON AND LT. COL. WATSON FROM AN- 
OTHER ANGLE 



COLONEL CARSON: . , A . .. , 

You say priority of fires to the 1st Battalion. Does that apply to both tne 
scheduled fires for the preparation and the ”on call” fires, after the attack 
jumps off? 



COLONEL WATSON: . 

Uh-huh’ (NODDING HIS HEAD) I want them to have all the firepower they need 
so that the 1st Battalion can push through in a hurry, once they jump off. 



CLOSE-UP ON LT. COL. CARSON. 



COLONEL CARSON: ^ 0 

Any specific targets you want to include in the preparation? 

CLOSE-UP ON COLONEL WATSON. 



COLONEL WATSON: 

Just a few. First, we’ve definitely located two aggressor OP’s. 

EXTREME CLOSE-UP ON MAP. (INSERT CAMERA) AS HAND POINTS OUT 
DETAILS. 



COLONEL WATSON’S VOICE: /nAnumir pt aqip to 

One here (POINTING FAR OUT ON MAP) and one ? er iL GT^THE 
ON MAP). At H-5, I want you to start smoking this OP (POINTING TO THE 
ONE FAR OUT) and continue to smoke it until we tell you to lift your fires. 



MEDIUM SHOT ON COLONEL WATSON AND LT. COL. CARSON AT MAP. 



COLONEL CARSON: . . A . 0 

Check! How about the other OP? Want us to take care of it, too? 

INDICATED IN OVERLAY 5 



COLONEL WATSON: 

Nope! I’ll handle it with my heavy mortar company. 
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COLONEL CARSON: 
Anything else? 



COLONEL WATSON: 

Yes, I've been up talking to the 201st, and there are two targets out there that 
are giving them a bad time. 

COLONEL CARSON: 

What are they? 

COLONEL WATSON: 

An anti-tank gun and some automatic weapons. 

EXTREME CLOSE-UP ON MAP (INSERT CAMERA) AS HANDS POINT OUT 
PLACES. 

COLONEL WATSON’S VOICE: (CONTINUING) 

The anti-tank gun is located here (PLACING CIRCLE ON MAP) and the auto- 
matic weapons are here (PLACING ANOTHER CIRCLE ON MAP). 

INDICATED IN OVERLAY 5. 

MEDIUM SHOT ON COLONEL WATSON AND LT. COL. CARSON. 

COLONEL CARSON: 

Good! 

COLONEL WATSON: 

One more thing: during the last two minutes of the preparation, I want you to 
place all the fire yoii can .... along this line, in front of my right Battalion. 

EXTREME CLOSE-UP ON MAP (INSERT CAMERA) AS HIS HAND DESCRIBES 
THE LINE, WITH GREASE PENCIL, ABOUT 1000 YARDS LONG. SHOULD RUN 
PARALLEL TO THE L.D. AND SHOULD BE ABOUT 300 YARDS IN FRONT OF 
L.D. (SEE OVERLAY 5) 

COLONEL WATSON: 

I want those men to know that the ground in front of them has been so well 
covered that they’ll have comparatively easy going for the first few hundred 
yards, and that plenty of artillery fire will be available to them whenever they 
need it. 

CLOSE SHOT ON COLONEL CARSON. 

COLONEL CARSON: 

Good idea! Initially, I can put two battalions on that mission. Probably be able 
to get general support artillery through Division Artillery, if we need it. 

MEDIUM SHOT ON COLONEL WATSON AND COLONEL CARSON. 

COLONEL WATSON: 

Fine! That’s about everything I can give you now. 



SCENE 33:: TERRAIN MODEL 



MEDIUM SHOT ON NARRATOR H. 



NARRATOR II: 

Colonel Carson will be coordinated with infantry through his artillery liaison 
officers, at the Infantry Battalion CP's. 

DISSOLVE: 

Test item 12P4 reproduced above was presented before the program in which 
priority of fires was treated, immediately after that program, and again one week 
later. Records were available on stratified samples for 438 officers and 299 enlisted 
men who had taken all three of the tests :n which this item appeared, and who had 
viewed the program in which the subject of priority of fires was treated. Table 11 
and Figure 9 report the percentages of correct responses given by officers and 
enlisted men on each of the three occasions when the item was presented. For the 
officers, there was a decrease of 7 per cent in the percentage of correct responses 
as a result of "instruction,” and a week later, there was a further decline of 4 per 
cent in percentage of correct responses. The enlisted men showed a negligible 
increase of 2 per cent as a result of instruction, and changed only slightly one week 

later. 
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25 



0 



INSTRUCTION 



Figure 9. The Adve » Effect of Sketchy Instruction upon Frequency of Correct Responses to 

Test Item 12P4 
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TABLE 11 



PERCENTAGES OF CORRECT RESPONSES GIVEN TO TEST ITEM 12P4 
BEFORE INSTRUCTION, AFTER INSTRUCTION, AND ONE WEEK LATER 



Before Immediately One Week 

N Instruction After Instruction After Instruction 



Officers 


438 


52.3 


45.0 


40.9 


Enlisted Men 


299 


30.8 


32.8 


32.4 



Data from this and other topics given sketchy treatments make it clear that a 
vague or sketchy treatment of a topic may not only fail to put across the point 
intended, but it may actually confuse some of the observers and cause them to 
acquire erroneous information. For example, before instruction, 229 officers - 
52.3 per cent of the sample - gave the correct response to item 12P4. Immediately 
after the program in which that topic was treated, 103 of the officers who had pre- 
viously given the correct response chose one of the incorrect alternatives. Of the 
209 officers who gave an incorrect response before instruction, only 71 gave the 
correct response after instruction. Results of testing other ineffectively taught 
topics showed interference effects from different portions of the same program or 
even from other programs in the series. 
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CONCLUSIONS 



In general, the results of this investigation indicate: 

1. That television instruction is an effective means of training large 
numbers of reservists dispersed in widely separated groups. All 
grades of personnel made statistically significant gains on test scores 
for each of the programs . 

2o That reservists not only learned from the television instruction, but 
they remembered most of what they had learned when re-tested four 
or six weeks later. 

3. That television instruction continues to be highly acceptable to the 
reservists after eight weekly sessions. Over 70 per cent of the 
officers stated that they preferred television instruction to conventional 
classroom teaching, and indicated that the television programs had 
been more instructive than the average training films they had viewed 
within the past two years. About 60 per cent of the enlisted men 
expressed the same preferences. 

4. That the amount of gain on test items is related to the explicitness of 
treatment of the topics on which these items are based. Items that 
are explicitly covered in the television sessions show large and sig- 
nificant gains. Some sketchily treated items show gains but many of 
them show losses in per cent of correct response after instruction 
indicating that trainees were confused rather than instructed by vague 
cr diffuse treatment of topics. 

5. That the type of instructional treatment given a topic influences the 
amount of learning. The various types of treatment recognized in the 
analysis are listed below in rank order of their effectiveness. Each of 
the first two types listed was more than twice as effective in producing 
learning as either of the last two types listed. 

a. Narration with meaning- conveying film 

b: T)rama with some form of narration 

c. Narration 

d. Narration with ’’atmospheric” film 

e. Drama 
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RECOMMENDATIONS 



RECOMMENDATIONS FOR USE 

It has been demonstrated that television can convey instruction effectively to 
laife'e and widely separated groups. It is recommended that demonstrational projects 
be undertaken immediately to provide service personnel with experience and with 
basic procedural patterns for utilising television in troop training. 

These demonstrational projects should be of four kinds: 

a. Distribution of televised lessons originating on a military base to 
numerous classrooms on the same base. 

Advantages: Security preserved 
Relatively low cost 

- Minimum scheduling difficulties 

b. Transmission by microwave of instructional sessions from commercial 
broadcasting studio to classes of troops on a single base. 

Advantages: Uses existing studio facilities 

Uses experienced television station personnel 
, Insures privacy but not absolute security 

c. Distribution of television lessons to classes of trainees over limited 
networks of commercial telecasting stations. 

Advantages: Reaches large numbers of trainees 
Uses existing facilities 
Insures well-prepared lessons 

d. Use of mobile equipment for remote pick-up and for fieJd exercises. 

Advantages: Makes unnecessary the transportation of large numbers 

of personnel to remote points 
Close-up views afforded all trainees 
Permits pick-up from areas dangerous for personnel 



RESEARCH SUGGESTIONS 

The production facilities employed in the ’•Command Post” series were compa- 
rable to those utilized in top flight network entertainment programs. Experiments 
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need to be undertaken to determine the training effectiveness of low-cost Production 
procedures using military instructors rather than professional actors. 
cost per man is not of primary importance if ten or twenty thousand trainees view 
a single production. However, production cost per man is of grea lmpor * 
only Lee hundred or five hundred trainees need to be reached by a given specialized 

te ““ the service schools of the Armed Forces should be surveyed to 
determine in which training programs television can apparently be used economically 
and effectively. On the basis of experience with the four types of exploratory 
demonstration^ television training projects recommended above, criteriashoul e 
developed so that any course of training could be "rated" as to 1 a ap a 1 1 
television. These criteria and rating procedures should then be validated expen- 

men *The data of this study have suggested that direct narration is much more ef- 
fective in producing learning than is dramatized action. However, other sources 
have indicated that the "straight lecture" type of instruction is relatively unsatis- 
factory. fnquiries need to be undertaken to determine the best balance between 

narration, demonstration, film and dramatized treatment. 

Carefully designed and well controlled experimental studies can be conducted 
only in those situations in which the director of the experimental investigation is 
given authority regarding the television lessons equal to that of the producer of the 

duction staffs is essential if "interest," "action," "expediency, “mehm.tations^ 
etc are to be prevented from interfering with sound experimental design. Som 
worthwhile studies can probably be conducted on a "permissive" or catch ' aS -^ C ‘ 
can" basis in connection with television instructional sessions where effo 

concentrated almost exclusively on production details However continued a ention 

must be given such studies to insure that the results being obtained justify 
expenditure of research funds being made. 



RESERVATION OF CHANNELS 

Television channels should be reserved for use by the Armed Forces sc^at 
they can initiate a program of military training by television. Some of these channels 
should be within the bands set aside for commercial telecasting. This would make 
possible the use of standard transmitting and receiving equipment. Other chann 
outside the commercial bands should be reserved for transmission of piogr 
where privacy is desirable. 
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Table 1 



Design of Experiment, 
Matched Groups, 2. 



Showing 1 . Intra-Pair Comparisons of Four Pairs of 
What They Saw, 3* Description of What They Saw 



Four Pairs of 
Matched Group 



What They Saw 



Description of What They Saw 

^ h 



A. 


First Pair 












Group One 


Kinescope 


They were told 
Kinescope 


it 


was a 




Group Two 


Kinescope 


They were told 
Training Film 


it 


was a 


. B * 


Second Pair 












Group One 


Training Film 


They were told 
Kinescope 


it 


was a 




Group Two 


Training Film 


They were told 
Training Film 


it 


was a 


C. 


Third Pair 












Group One 


Training Film 


They were told 
Kinescope 


it 


was a 




Group Two 


Training Film 
in Color 


They were told 
Training Film 


it 


was a 


D. 


Fourth Pair 












Group One 


Training Film 


They were told 
Kinescope 


it 


was a 




Group Two 


Training Film 
in Color 


They were told 
Kinescope 


it 


was a 
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FOREWORD 



Television as a mass communication medium has been 
accepted widely by the American people* Special Devices 
Center has tested the new medium for potential uses in 
military training. Out of these television tests has 
come a new kind of training film — the kinescope. The 
kinescope, because it is made with television equipment, 
can be made more quickly and cheaply than standard train- 
ing films. 

The kinescope , however , has appeared to have some 
limitations for military training. For one thing the 
film quality of the kinescope seems somewhat inferior 
to the standard training film. For another the speed 
with which the kinescope is made precludes the profes- 
sional slickness which characterises many standard films. 
Are these things serious limitations and do they affect 
military training? 

This study provides one answer to the problem of 
the effectiveness of the kinescope for training. ^ The 
need fdr cheap, mass-produced training films indicates 
the importance of an evaluation of the kinescope for 
military training. 




Robert Jackson/ / 
Applications R<tacargn i 
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LEARNING FROM KINESCOPES AND FILMS 
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SUMMARY 

Introduction 

The purpose of this study was to discover 
why kinescopes, in spite of some obvious limitations, 
were doing such a good job of military training. 
Previous experiment indicated that kinescopes were 
teaching as effectively or even better than regular 
classroom instruction. The learning results from 
kinescopes were surprising because experience with 
standard training films had led to the opinion that 
such films were about equivalent to regular class- 
room instruction or slightly inferior. 

A question xras raised whether kinescopes, 
associated as they are with the television industry, 
might account for the superior results. In other 
words, did kinescopes, simply because they were kine- 
scopes . attract a special interest and readiness to 
participate in the learning situation? 

From this general question two specific ques- 
tions were asked in this study: 

1. Would lear ni ng by kinescope be less 
if the kinescope were described as 
a standard training film ? 
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2. Would learning by standard training 
film be better if the film were 
described as a kinescope? 

Procedure 

The experiment was designed to evaluate 
the learning of four pairs of matched groups: 

1# First Pair : 

One group saw a kinescope which 
was described as a kinescope. 

The other group saw the same 
kinescope which was described to 
them as a standard training film. 

2. Second Pair : 

One group sa w a standard training 
film which was described to them 
as a kinescope. 

The other group saw the same 
standard training film which was 
described as a standard film. 

3 * Third Pair : 

One group saw th^ standard training 
film which was described to them 
as a kinescope. 



The other groixp saw the same 
standard training film in color 
which was described as a standard 
film. 

lj. # Fourth Pair : 

One group saw the standard training 
film which was described to them 

as a kinescope. 

The other group saw the same 
standard training film in color, 
which was also described to them 
as a kinescope. 

Results 

1. First Pair : 

The group which was told they were looking 
at a kinescope learned more than the group which was 
told they were looking at a standard training film 
when a kinescope was used in both instances. 

2. Second Pair : 

The group which was told they were looking 
at a kinescope l earned more than the group which was 
told they were looking at a standard training film 
when a standard film was used in both instances. 
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3. Third Pair : 

The group which was told bhey were looking 
at a kinescope learned_m ore than the group which was 
told they were looking at a standard training film 



in color * 



b. Four th Pair : 

The group which was told they were looking 

at a kinescope 3 ^rneJ_aho^,th^am ® s S roup 

which was told they were looking at a kinescope in 



color* 



Re c omm e nd a t ions 



Two recommendations are offered 



from these 



results: 

1. Kinescopes in 

i imitations should, h e^on^gj^a^^^ 

for lit a rv trai ning.* 



2. unless it is essentia l to the 

subject bei n g taught, does not^n c r eas e the effectiv e 
hags of television training sessions. 
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LEARNING FROM KINESCOPES AND FILMS 



INTRODUCTION 

1. What Are Kinescopes? 

Kinescopes are film recordings of television 
programs • Kinescopes are made by recording the image 
from a television tube on movie film* Experimental 
training kinescopes made at Special Devices Center , 
Office of Naval Research, are recorded on 16mm film* 
When projected on a screen, these kinescopes appear 
much like standard l6mm films* 

2* What Is The Problem In Using Kinescopes ? 

Although kinescopes appear in general like 
standard films, they have some important limitations. 
For one thing, the film quality of kinescopes is some- 
what inferior to standard films. Picture definition 
is inferior, details are fuzzier, and the striking 
clarity of itfell-made films is generally missing. 

Persons who have experimented with kinescopes 
have been puzzled as to whether the inferior film 
quality would interfere with learning. Obviously, deg- 
radation in film quality could be carried to a point 



where training might be seriously impaired. An impor- 
tant problem has been whether the average kinescope is 

suitable for military training. 



% 

3 • What Ar^ The Previo i~is__Res33lts?. 

When kinescopes were l sed in SDG experiments 
on mass training, good results wore obtained. 1 Kine- 
scopes were found to be almost equal to live television 
for military training. Kinescopes also proved superior 

to regular classroom instruction. 

These are surprising results. Previous 
experiment with standard training films showed them 
to be about comparaole to regular classroom instruc 
tion. Kinescopes seemed to be standing up as well, 
if not better, than standard training films for mili- 

tary instruction. 



k # Whv Are Kinescopes So 






This study sought an explanation of the 
excellent test results that had been obtained in 
learning hy kinescopes. Comparison of the kinescopes 
that had been used with standard training films showed 



*^SDC Technical Report No. 4?6-02-2. 
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that the kinescopes were unexceptional. They were 
inferior in film quality. Content of these kinescopes 
revealed nothing special. Format, learning organiza- 
tion, instructor techniques, etc., were not superior to 

training films. 

At this point we suspected that kinescopes 
might be training so effectively simply because they 
were kinescopes. In other words, we guessed that the 
method of making kinescopes, associated as it was 
with the television industry, might account for the 
su-k °ior learning. Kihe scopes must attract a special 
interest and readiness to participate in the learning 

situation, it was thought. 

Thus two basic questions were asked in 

this study: 

A. Would learning by Icinescop e ^ be 
less if the kinescope were 
described as a standard trAinjgS. 
film? 

B . Would learning by standa rd train- 
ing film be bett er if the film 
were described as a kinescope? 
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EX PER IMENT AL DESIGN AND PROCEDURE 

1# TTnw Was The ExperimenA ^egJ^B^ ? 

The experiment was designed to evaluate 
the learning of four pairs of matched groups. (See 
Table I) . 

A. First Pair ; 

One group saw a kinescope 

which was described as a kinescope. 

The other group saw the same 
kinescope which was described as 
a standard training film. 

B* Second Pair : 

One group saw a standard 
training film which was described 

as a kinescope* 

Tbe other group saw the same 

standard training film which was 
described as a standard film* 

C* Third Pair ; 

One group saw the standard 
training film which was described 
as a kinescope* 
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Table 1 



y-r 

( 

X 



Design of Experiment, Showing 1. Intra-Pair Comparisons of Pour Pairs of 
Matched Grcmps, 2. What They Saw, 3* Description of What They Saw 



Four Pairs of 
Matched Group 



A. First Pai r 
Group One 

Group Two 

B. Second Pair 

r 

Group One 
Group Two 

C* Third Pair 
Group One 

Group Two 

D* Fourth Pair 
Group One 

Group Two 



What They Saw 



Kinescope 

Kinescope 



Training Film 
Training Film 



Training Film 



Training Film 
in Color 



Training Film 



Training Film 
in Color 



Description of What They Saw 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



They were told it was a 
Kinescope 

They were told it was a 
Kinescope 



The other group saw the same 
standard training film in color 
which was described as a standard 

film. 

D. Fourth Pair : 

One group saw the standard 
training film which was described 

as a kinescope. 

The other group saw the same 
standard training film in color 
which was also described as a 
kinescope. 

Who Were The Subjects? 

The subjects were 2*+0 Airmen stationed at 
Mitchell Air Force Base, Garden City, New York. 

The Airmen were divided into four pairs of matched 
groups, each group containing thirty men. 

The Airmen were matched as closely as possi- 
ble on the following factors: 

A. Age 

B. Rank 

C. Classification scores (intelligence 
and aptitude) 

D. Years of schooling 
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The other group saw the same 
standard training film in color 
which was described as a standard 
film. 

Fourth Pair : 

One group saw the standard training 
film which was described to them 
as a kinescope. 

The other group saw the same 
standard training film in color 
which was also described to them 
as a kinescope. 

Results 

1. First Pair : 

The group which was told they were looking 
at a kinescope learr d more than the group which was 
told they were looking at a standard training film 
when a kinescope was used in both instances. 

2. Second Pair : 

The group which was told they we re looking 
at a kinescope learned more than the group which was 
told they were looking at a standard training film 
when a standard film was used in both instances. 



Ill 



3 * Third Pair s 



The group which was told they were looking 
at a kinescope learned more than the group which was 
told they were looking at a standard training i ilm 

in color ♦ 
h* Fourth Pair : 

The group which was told they were looking 
at a kinescope learned about th e sa me, as t-ne group 
which was told they uqtq looking at a kinescope in 

color • 



Reco mmendations 

Two recommendations are offered from these 

results: 

1 0 Kinescopes in suite of their technical 
limitations should he co nsidte^ 
fo r military trai ning.* 

2. Color « unless it is essential to the 
subject being taught t does not increa se the effective 
ness of television training sessions* 
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LEARNING FROM KINESCOPES AND FILMS 



INTRODUCTION 

1. What Are Kinescopes? 

Kinescopes are film recordings of television 
programs. Kinescopes are made by recording the image 
from a television tube on movie film. Experimental 
training kinescopes made at Special Devices Center, 
Office of Naval Research, are recorded on 16mm film. 

When projected on a screen, these kinescopes appear 
much like standard l6mm films. 

2 . What Is The Problem In Using Kinescopes? 

Although kinescopes appear in general like 
standard films, they have some important limitations. 
For one thing, the film quality of kinescopes is some- 
what inferior to standard films. Picture definition 
is inferior, details are fuzzier, and the striking 
clarity of well-made films is generally missing. 

Persons who have experimented with kinescopes 
have been puzzled as to whether the inferior film 
quality would interfere with learning. Obviously deg- 
radation in film quality could be carried to a point 



where training might he seriously impaired. An impor- 
tant problem has been whether the average kinescope is 

suitable for military training* 

% 

3 . i-Jhnt Are The Previous Results? 

When kinescopes were used in SDC experiments 
on mass training, good results were obtained . 1 Kine- 
scopes were found to be almost equal to live television 

for military training. Kinescopes also proved superior 

*? 

to regular classroom instruction. 

These are surprising results. Previous 
experiment with standard training films showed them 
to be about comparable to regular classroom instruc- 
tion. Kinescopes seemed to be standing up as well, 
if not better, than standard training films for mili- 
tary instruction. 

1,.. Why Are Kinescopes So Good. liLlli li . bary . T raining 

This study sought an explanation of the 
excellent test results that had been obtained in 
learning by kinescopes. Comparison of the kinescopes 
that had been used with standard training films showed 
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that the Kinescopes were unexceptional. They were 
inferior in film quality. Content of these kinescopes 
revealed nothing special. Format, learning organiza- 
tion, instructor techniques, etc., were not superior to 

training films. 

At this point we suspected that kinescopes 
might be training so effectively simply because they 
were kinescopes. In other words, ve guessed that the 
method of making kinescopes, associated as it was 
with the television industry, might account for the 
superior learning. Kinescopes must attract a special 
interest and readiness to participate in the learning 

situation, it was thought. 

Thus two basic questions were asked in 

this study: 

A. Would le arning by kinescope_be 
ip.qs if th e kinescope were 
described as a standard l^ining, 
film? 

B. Would learning by s tan kard , train- 
in g film be better if the_f iM 
were described as a kinescope? 
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EXPERIMENTAL DESIGN AND PROCEDURE 



1 # How Was The Experiment, Designed? 

The experiment was designed to evaluate 
the learning of four pairs of matched groups. (See 
Table I). 

A. First Pair s 

One group saw a kinescope 

which was described as a kinescope. 

The other group saw the same, 
kinescope which wa s described as 
a standard training film* 

B* Second Pair : 

One group saw a standard 
training film which was described 

as a kinescope* 

The other group saw the same 
standard training film which was 
described as a standard film. 

C. Third Pair : 

One group saw the standard 
training film which was described 
as a kinescope* 




Table 1 



Design of Experiment, Showing 1. Intra-Pair Comparisons of Four Pairs of 
Matched Groups, 2. What They Saw, 3* Description of What They Saw 



Four Pairs of 
Matched Groups 



What They Saw 



Description of What They Saw 



A. First Pa ir 
Group One 



Group Two 



Kinescope 

Kinescope 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



B. Second Pair 
Group One 

Group Two 



Training Film 
Training Film 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



C. Third Pair 
Group One 



Group Two 



Training Film 



Training Film 
in Color 



They were told it was a 
Kinescope 

They were told it was a 
Training Film 



D. Fourth Pair 
Group One 



Group Two 



Training Film 



Training Film 
in Color 



They were told it was a 
Kinescope 

They were told it was a 
Kinescope 




The other group saw the same 
standard training film in color 
which was described as a standard 

film. 

D* Fourth Pair : 

One group saw the standard 
training film which was described 

as a kinescope. 

The other group saw the same 

standard training film in color, 
which was also described as a 
kinescope. 

2. Who Were The Subjects? 

The subjects were 2^0 Airmen stationed at 
Mitchell Air Force Base, Garden City, New York. 

The Airmen were divided into four pairs of matched 
groups, each group containing thirty men. 

The Airmen were matched as closely as possi- 
ble on the following factors: 

A. Age 

B. Rank 

C. Classification scores (intelligence 
and aptitude) 

D. Years of schooling 
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3 How Was Learning Estimate d? 



Learning was estimated by administering a 
100 item true-false test. Two tests were constructed. 
One was based on the content of the kinescope. The 
other was based on the content of the standard train- 
ing film. 

Both questionnaires were pre-tested and 
standardized by analyzing the results of the pre-tests. 
The pre-tests were conducted among undergraduate 
college students in the New York Metropolitan area. 

l+ # How Were The Kinescope And Film Selected? 

The results of this experiment are based 
on intra-media comparisons, that is the kinescope 
against itself and the standard training film 
against itself. Both a kinescope and a film were 
used in the experiment simply to validate the re- 
sults. It was felt that the validation would be 
better if the kinescope and film were fairly similar. 

For this reason three criteria were applied 

in the selections: 

A. The content of both should be 

similar. 



B. The viewing tine of both should 
bo the same. 

C. Both should be capable of being 
plausibly described either as a 
kinescope or a standard training 
film. 

The kinescope selected was '’Radiological 
Defense". This is an experimental training kine- 
scope made by Special Devices Center to train mili- 
tary personnel in atomic defense. The film was 
"Patterns for Survival". This is a commercial edu- 
cational film made to train civilians in atomic 
defense. 

Both mot the three selection criteria 
indicated above. 

5 . How Was The Experiment Conducted? 

The following procedure was followed with 

all of the experimental groups: 

An explanation of the true-false 
test was read. 

B. The true-false test was given. 

C. A description of the presentation 
as a kinescope or a standard 



1 



s -- 

» 

v training film was read (See Design, 

Table 1) • 

D. The kinescope or training film was 
shown. 

E. The original true-false test (step 
B) was given again. 

The important variable in the experiment 
is what the groups were told they were looking at 
(a kinescope or a standard training film) • In order 
to keep a group from being unduly impressed, we 
attempted to make our description of the kinescope 
as matter-of-fact as possible. In other words, we 
tried not to overdescribe the kinescope in contrast 
to the standard training film. 

The groups which were told they were look- 
ing at a kinescope heard the following description: 

”Yqu are about to see a kinescope. A kine- 
scope is a television program that is filmed through 
a television camera. The television program which 
you will see was broadcast from the Special Devices 
Center to train people in atomic defense. We are 
trying to find out how well this television program 
trains.” 
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The groups which were told they were looking 
at a standard training film were read a similarly 
brief and matter-of-fact description of the film. 

No time limit was imposed on the true -false 
test. Average group time for the test was about 
twenty minutes. If a subject had trouble with the 
mechanics of the test, he was given unobtrusive 
assistance. No other assistance was given. Each 
group took about 1 hour and 20 minutes to complete 

the experiment. 

Two test scores were obtained for each 
subject, the "before" score and the "after" score. 
These scores were corrected for chance by subtract- 
ing the number of wrong answers from the number of 
rights. Final scores were determined by subtracting 
the corrected "before" scores from the corrected 
"after" scores. All data in this repprt are based 
on these corrected gain scores. 

RESULTS 

1 • Were The Groups Well Matched? 

Scores on the "before" tests were examined 
to determine whether the paired groups had been 
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successfully matched. The following results indi- 
cated that the paired groups had been evenly matched 

A. Mean group scores on the "before" 
tests did not differ significantly 
within the paired groups# Prior 
information about atomic defense 
was almost the same in the paired 
groups . 

E# Variance scores on the "before" 

tests did not differ significantly 
within the paired groups# Distri- 
bution of scores within the paired 
groups was thus about the same 
before seeing one of the films. 



2. Did Learn in g Differ 

All of the experimental groups learned from 
the kinescope or training film as indicated by im- 
proved "after" scores. Where expected, the amount 
learned differed significantly within the paired 



groups. 

By examining the gain scores of the paired 
groups, the following was found; 
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A. First Pair : (Saw Kinescope) 

The group which was told they were 
looking at a kinescope learned more 
than the group which was told they 
were looking at a standard train- 
ing film. 

This difference was significant at 
the 1 percent confidence level. (See 
Figure 2) • 

B. Second Pair : (Saw Training Film) 

The group which was told they were 
looking at a kinescope learned more 
than the group which was told they 
were looking at a standard train- 
ing film. 

This difference was significant 
at better than a 1 percent confi- 
dence level. (See Figure 3). 

C. Third Pair : (Saw Training Film) 

The group which was told they were 
looking at a kinescope learned more 
than the group which was told they 
were looking at a standard train- 
ing film in color . 
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FIGURE 2 



Results of First Pair of Matched Groups Viewing the Same 

Kinescope under Two Conditions 



Group One Was Told They Were 
Looking at a Kinescope 




Result: Learning Score = 36 



Group Two Was Told They Were 
Looking at a Training Film 




Result: Learning Score = 28 



Superior Learning of Group One Is Significant. 

(It Could Have Occurred by Chance about One Time 
i n a Hundred.) 



figure 3 



Results of Second Pair of Matched Groups Viewing the Same 
Training Film under Two Conditions 



Group One Was Told They Were 
Looking at a Kinescope 



Group Two Was Told They Were 
Looking at a Training Film 




Result: Learning Score = 40 



Result: Learning Score = 26 



Superior Learning of Group One Is Significant. 

(It Could Have Occurred by Chance Less Than One Time 
in a Hundred.) 



figure b 



Results of Third Pair of Matched Groups Viewing the Same 
Training Film (One Version in Color) under Two Conditions 



Group One Was Told They Were 
Looking at a Kinescope 



Group Two Was Told They Were 
Looking at a Training Film 




Result: Learning Score = 40 



Result: Learning Score = 25 



Superior Learning of Group One Is Significant. 

(It Could Have Occurred toy Chance Atoout One Time 
in a Hundred.) 



FIGURE 5 



Results of Fourth Pair of Matched Groups Viewing the Same 
Training Film (One Version in Color) under the Same Conditions 



Group One Was Told They Were 
Looking at a Kinescope 




Result: Learning Score = 40 




Group Two Was Told They Were 
Looking at a Kinescope 




Result: Learning Score = 34 



Learning of the Two Groups Does Not Differ Significantly. 

(It Could Have Occurred by Chance about Thirty Times 
in a Hundred.) 




This difference was significant 
at the 1 percent confidence level. 
(See Figure W . 

D. Fourth Pair : (Saw Training Film) 

The group which was told they were 
looking at a kinescope learned 
about the same as the group which 
was told they were looking at a 
kinescope in color . 

The slight difference which occurred 
in favor of the first group was 
considered insignificant. (See 

Figure £) #* 

3 # What Are The Conclusions ? 

The two most important conclusions from this 
study ares 

A. When a kinescope or a s tandard 
training film is described as_a 
kinescope « learnin g increase^ 
significantly . 

B. The superior le arning resulting 
from describing the film as a 
kinescope occurs whether o r not 
the presentation is in colo r* 
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if. What Are The Inferences? 

No interpretation should he drawn from these 
results that standard training films should he called 
kinescopes to improve learning. The deception prac- 
ticed in this study was done simply for scientific 
validation. An accurate statement of the facts was 
given to the subjects when the experiment was completed. 

The conclusion that color in a training film 
has no effect on learning is limited hy the condi- 
tions of the experiment. It is important to note that 
in the film used, color was not relevant to the subject 

which was taught. 

It is not easy to account for the results 
of this experiment. Three possible explanations are 

offered: 



The novelty of kinescopes may quicken the 
interest of trainees and create a greater readiness 
to learn. Novelty of kinescopes , associated with 
novelty of television, may thus explain differences 
in learning obtained in this study. It should be 
noted that the subjects were from the Metropolitan 



A. Novelty effect 

B. Immediacy 

C. Culture 
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New York area and familiar with television. This may 
not mean that they were similarly familiar with kine- 
scopes* 

The impression of immediacy associated v/ith 
television may he carried over to kinescopes. This 
is to suggest that an impression of recency or im- 
mediacy will increase trainees* readiness to learn. 

This effect can he compared to the impression which 
the word "Extra** makes on the daily edition of the 
newspaper . 

A third explanation suggests that, these 
results are due to a cultural characteristic oi Ameri- 
cans. This cultural characteristic has hoen described 
hy Margaret Mead 1 and others as a desire to change in 
certain approved directions. This pattern of change 
is considered institutionalized and a predictable 
factor in American culture. Technological progress 
represented hy television and hy kinescopes would, 
in this thinking, be an acceptable direction of change. 
Because they represent progress, kinescopes are thus 
a more effective medium for teaching. 

5 . What Are Th<=> Recommendation s? 

A* Kinescopes in spit e of their. 



^ead, Margaret. And Keep Your Powder Dry* 



— lk— 



technological limi tations should 
be considered very satisf actory 
for military training. * 

Color, unless it is essential tc> 
the subject being taught* does n<*t 
increase the effectiveness of tele- 
vision training sessions * 
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Problem 

Certain instructions given prior to a film showing have been shown 
to increase learning (20-TV-l and 269-7-2U). The 20-TV-l study- 
showed that if two groups were shown the same film, the group told 
it was looking at a kinescope learned significantly more than the 
group told it was looking at a film. This study was done to check 
this finding now that television and kinescope recordings are no 
longer a novelty. 



Results 



This study found that trainees learned about the same amount from a 
film whether they were told it was a kinescope or a film. Apparently 
the novelty of kinescopes no longer stimulates trainees to greater 
learning. 



Recommendation 

No advantage in learning can be expected by calling a film a tele- 
vision recording. This change in the facts as compared with the 
earlier report is thought to be associated with the idea that tele- 
vision is no longer a novel situation. These two reports point up 
the marked but apparently temporary value of novelty in the learn- 
ing situation— an aspect of learning which is little understood. 
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